The businessman has called for a general meeting through his MASH Holdings company, which owns nearly 9% of the total shareholding in the Ibrox side.
Ashley's Sports Direct gained security over various club properties, as well as its trademarks, in return for the loan provided in January for emergency working capital.
The Newcastle United owner subsequently lost control over Rangers' affairs when his associates, Derek Llambias and Barry Leach, were voted off of the club's board in favour of Dave King, Paul Murray and John Gilligan coming to power in March.
Now Ashley wants his money back, calling the bluff of majority shareholder King who has previously stated he would wait to review the details of the deal before deciding whether to repay under the terms of the agreement.
A spokesman for Rangers told STV: "The board have received the request and are considering it with our advisers."
If the current board can find the £5m required, it would ensure Ashley no longer had control of Rangers' training ground Murray Park, the Albion Car Park and their Edmiston House property at Ibrox Stadium.
It would also see him relinquish his stranglehold over the club's brands and trademarks, as well as guaranteeing a greater return of retail profits from the joint venture between Rangers and Sports Direct.
In turn, Rangers would also be free to take out a future loan secured against Ibrox should the board so wish. At present, the terms of its finance deal means Sports Direct must give consent before any security can be granted.
Ashley has also posed several questions of the club's board to be addressed at a general meeting.
In his letter to the directors, dated April 29 this year, he calls for clarity on the club's delisting from the Alternative Investment Market and the subsequent failure to date for the company's shares to be listed on an alternative exchange.
Ashley also wants shareholders to be given full details on a £1.5m loan given by Douglas Park, George Letham and George Taylor in March.
Rangers have until May 20 to call a general meeting, which must subsequently be held within 28 days of the board convening the meeting.