The Welsh club are searching for a new boss following Paulo Sousa's appointment at Leicester.
Speed is on the coaching staff at Bramall Lane under manager Kevin Blackwell.
Swansea chairman Huw Jenkins: "As was the case with a few managers, an approach was made for Gary. It was turned down though and, once that happens, there is nothing we can do about it.
"That is why you have several options. It is not a disappointment as such because I have done this job long enough to know that you can't just focus on one target.
"Gary was only one of a number of options we were looking to speak to, to see what their thoughts were."