In 1891 they were keeping a boarding house at 24 Derby Road.
Although house numbers are not given, Derby Road appears in the 1861 census.

This looks like your family. You should have no problem geting birth/marraige details.

1881, rg11/3077/88 page 2
26 sheep Street
Rugby, Warwick
Alfred Tongue, head, married, 38, boot maker, Rugby
Sarah, wife, married, 31, Rugby
Fanny M, daughter, 9, Rugby
Mary H, daughter, 7, Rugby
Grace E, daughter, 3, Rugby
Constance J, daughter, 1, Rugby
Alfred W, son, 3 months, Rugby