In 1209, some residential and military structures already existed here, the latter with extensive visual control of the valley. In the Middle Ages, the difficulties of repopulating the area resulted in the royal attribution of charters. This guaranteed the defence of the border under pressure from the Kingdom of León, which held the extensive bayou of the fortified village of Sabugal. The charter was granted in 1228 by King Sancho II, when the castle, or citadel, was built. The latter corresponds to a Romanesque-Gothic rock castle, which underwent occasional interventions during the Manueline period, while the walls, somewhat prior to the castle, were built using two strong parallel walls as frames, between which the space was filled with large stones and gravel. The citadel, located on the southern edge of the walled perimeter, has a keep in the centre of the enclosure. With a square plan, it is solid up to the door and has three arrow slits. A curiosity is that the board game, nine men’s morris, is carved on the slab at the base of the tower. There is also a cistern inside the castle. Access to the interior of the castle is through a door with a basket-handle arch, partly excavated in the outcrop. On top of it, and as defensive reinforcement, there is a walkway with machicolations and, beside it, the royal weapons of King Manuel I with armillary spheres can be seen. Also worth mentioning here are the troneiras (circular openings in the battlement to support bombards, a piece of light artillery). This walkway is popularly called the “Pilate’s Balcony”. The walled perimeter of the village has an irregular oval outline. In several places, the wall rests directly on the rocky outcrop and does not have merlons, contrary to what should occur in a castle, with only those in the keep having been preserved. It has four entrances connecting it to the outside: the Town Gate or Municipal Gate, the New Gate, the False Gate and the small Betrayal Gate, located in the castle. In addition to the Keep, there is another tower, the Beam Tower, with a square floor plan, devoid of openings. Among the castle's particularities, the absence of merlons on the walls, and the “pole” and “cubit” (standard measures) carved in the New Gate are noteworthy. These ancient medieval measurements, the cubit (67 cm/3 hands) and the pole (110 cm/5 hands) were carved here so that everyone visiting the fair would use the same standard measurement. During the Modern Period (17th-18th centuries), the outskirts of the walled village began to develop, initially along the path to Sabugal. Later, the village spread out along the slopes and moved away from the walls. It developed relatively far from the walled nucleus and still retains some manor houses. Napoleonic troops passed through Sortelha and dynamited part of the castle wall, following the battles they waged there. Thus, from the Corro Square, facing the Town Gate, it is possible to see, to the left of it, a part of the recently reconstructed wall. The Castle has been classified as a National Monument since 1910 and this ancient village was recently included in the Historical Villages of Portugal programme.

The building housing the Old Town Hall and Prison is located in the Pillory Square and its construction dates back to the 16th century, considering the Manueline weapons and the dating of the pillory. The building has two floors, the lower floor being the jail, where the original latrine has been preserved, and the upper floor being the Town Hall. The floor plan, of great simplicity, has two compartments per floor. The openings are made with straight lintels without a protruding frame. Interestingly, the lower floor incorporates the city wall into its structure. In 1855, with the extinction of the municipality of Sortelha, the building was transformed into a primary school, recently closed. Currently, it is used as the offices of the Sortelha Parish Council.

It is said that one night, a host of Moors surrounded the fortress of Sortelha, to take back that strategic site. In the castle were the mayor, his wife and their daughter, a beautiful maiden. - "What a beautiful man!" she exclaimed, following him with her eyes - "What a beautiful woman!" thought the Moor from the top of the walls. From that moment, neither one nor the other could stop thinking about meeting. Therefore, the Moorish leader promised freedom to a Christian prisoner, with the conditions he imposed: - "I will speak alone with the maiden and you will open the door for her." The girl's mother, who was said to be a witch, realised that something was afoot that night. The lovers were by the walls, kissing. - “Damn you! I curse you and turn you to stone!” - shouted the sorceress. And the two lovers, embracing, lips to lips, were turned to stone. And so they remained, forever.

Bracejo (Brachypodium phoenicoides) is a filiform plant which grows in clumps, in dry terrain at medium altitude levels. They are a very popular food for ruminants, especially cattle. Since ancient times, it has been used for weaving. Bracejo grows wild in the countryside and is collected by artisans for use as a raw material. With the help of a needle and raffia, the bracejo is weaved into the most varied objects for domestic and decorative use, namely, mats, baskets, various bases and whatever the imagination can create. It is also still commonly used for brooms.

In this region with a treeless and rugged landscape, bizarre-looking granite formations abound, which, shaped by wind and rain, acquire curious appearances with fanciful connotations decorating and brightening the scenery surrounding Sortelha. Many of them have suggestive names, and others are associated with some local legends. In Sortelha, the famous “Old Woman’s Head” stands out because of its appearance, reminiscent of a face aged by time.

The medieval walkway preserved from the New Gate, on the outside of the walls, crosses the complex which includes the Old Hospital and Church of Mercy, passing through the Chapel of Santiago, extending over a section approximately 300 m long and 5 m wide . The paving of this section was carried out with regular granite stones and probably similar to those identified along Rua Direita.