Though Doctor Robert Richardson was assigned to Fort St. Joseph with the Queen's Rangers, his wife, Madeleine (Askin) Richardson, a member of Detroit's (and Canada's) well known Askin family, and their children (including Major Richardson), stayed in Detroit with her father.
In the fall of 1801 a detachment of the Queen's Rangers was ordered to Fort St. Joseph, a post on the island of the same name, near the head of Lake Huron. Dr. Richardson* accompanied this force to the western post, but the prospects of providing suitable accommodation for his wife and young family in this fort were not very promising, so it was arranged that Mrs. Richardson and family should live with her father at Detroit.
*...a young Scotchman named Robert Richardson, the Assistant Surgeon, a scion of the younger branch of the Annandale family, which had clung to the fortunes of the Pretender. The young military surgeon became acquainted with the leading merchant of the place, Honorable Robert Hamilton, member of the Legislative Council, who had married Catherine Askin, daughter of Colonel John Askin, a wealthy merchant of Detroit. At his home Dr. Richardson met Miss Madeleine, another daughter of Colonel Askin, then on a visit to her sister. ...in the records of St. Mark's church, Niagara, that "Doctor Robert Richardson, bachelor , and Madeleine Askin, spinster," were married...on January 24th, 1793.