He is a liveryman and past Prime Warden of the Worshipful Company of Fishmongers of the City of London, Chairman of the Governing Body of Gresham's School, Holt, Vice-President of the Anglers’ Conservation Association, Trustee of the Country Land and Business Association Charitable Trust, and Trustee of the Rudhams Playing Fields Trust. In 1992, Duckworth-Chad was appointed High Sheriff of Norfolk. He now serves as a Deputy Lieutenant of Norfolk.
He is the son of Anthony John Stanhope Duckworth, the son of George Herbert Duckworth and Margaret Leonora Evelyn Selina Herbert. Anthony Duckworth-Chad's grandfather, Sir George Duckworth (1868–1934), was a half-brother of the painter Vanessa Bell and the writer Virginia Woolf (the latter charged George and her other half-brother, Gerald, with molesting her as a child and teenager). His great-grandmother, born Julia Prinsep Jackson, was a niece of Julia Margaret Cameron, the photographer, while his great-grandfather, Herbert Duckworth, was a barrister. After her husband's death, Duckworth-Chad's great-grandmother Julia married secondly the author Leslie Stephen. Duckworth-Chad's paternal grandmother, Lady Margaret Herbert, was a daughter of the 4th Earl of Carnarvon, making him a cousin of Auberon Waugh.
Duckworth-Chad's great-uncle Gerald Duckworth founded the London publishing firm of Duckworth & Co. Duckworth-Chad agreed to become the heir of his great-uncle's widow with the stipulation that he would add her maiden name, Chad, to his own.
In 1970, Duckworth-Chad married (Elizabeth) Sarah Wake-Walker, a granddaughter of the seventh Earl Spencer. They have three children, James Anthony de L'Etang Duckworth-Chad (born 1972), William George Christopher (1975) and Davina Alice (1978).
Duckworth-Chad's wife, Elizabeth Wake-Walker, is a first cousin of Diana, Princess of Wales, and he is also reported to be a close friend of Charles, Prince of Wales. His son James was Equerry to The Queen from 2001 to 2004, and his daughter Davina is a friend of her second cousins The Duke of Cambridge and Prince Harry, both of whom attended her marriage to Tom Barber at Pynkney Hall on 25 September 2004.