William Clarke was born on May 23, 1858. He grew up in Raymond, New Hampshire, a small town just outside Manchester. William’s parents and aunt were servants for a wealthy family that was part of the New England aristocracy. William’s mother Ruth, was the personal maid for Mrs. Anne Richardson, his father Jack was the head butler, under the supervision of Anne’s husband Thomas, and his aunt Mary was the head cook. William was lucky to grow up in the Richardson’s home – Anne and Thomas loved the Clarkes, and they treated them well. Anne and Thomas were older when William was born, and their children were already grown, so they loved having a child around again, and they included William and his parents as part of their family.

In his years living with the Richardson’s, William learned many values and skills. He was raised Catholic, and he read passages from the Bible every evening with Anne. His family taught him home skills, like manners, butlery, cooking, and horse grooming. But Thomas taught William his most important skills: from Thomas, William learned to read and write, and moreover, to express his opinions through public speaking and debate. Having spent his childhood in the loving and supportive environment that his parents and the Richardson’s created, William left home after his parents died, bound for New York City at the age of twenty-three and unaware of the harsh, prejudiced reality he was to face there.
When William reached the city, he couldn’t find a job as a servant, or as a cook, but he answered an advertisement for a ship crew in the harbor. William worked on this crew for the next six years, during which time he found a room to rent at 280 Bowery Street, for only $2 per month. He was doing well on his own, and he went home to New Hampshire every Christmas to spend the holidays with his Aunt Mary, who now worked for a friend of the Richardson’s living in Manchester. However, one day in November of 1887, the city council proposed a bill cutting back on commerce activity in the harbor, which threatened William’s place on the crew.

On December 6, the council put the bill up for referendum voting by the public. Because he was black, William was not allowed to vote on this type of municipal bill, but in his years with Thomas, William had learned never to silence his voice if he had something important to say. So William went to the voting center and attempted to vote against the bill. He was stopped by the police, arrested, and sentenced to one to ten years in Sing Sing prison for attempted illegal voting.