Newcastle set a club record of 14 away wins on their way to winning the Championship title last season.
But their Swansea success - secured by skipper Jamaal Lascelles' second-half header - was their first in the Premier League since a 2-1 victory at Tottenham in December 2015.
"We had to be in the Championship and it was a tough year," Perez said.
"Nobody feels good when you get relegated, but we have been working very hard to be in this league.
"We want to show that we should be playing in this league and these wins will give us some confidence."