People often talk about how much places you knew in childhood shrink as you grow up and it’s true, of course they do; wiser, perhaps, more widely travelled, taller, you have a different perspective on things. But the other reason they appear to shrink is, surely, because you did know them, and the past comes crowding in on you; there are buildings still visible to you though they vanished long ago, your former selves jostle for attention at your elbow. If I walk through Rochester now I find twenty of me already there beforehand. There I am in the Vines, standing on one of the trees that blew down in the storm of ’87. There I am, walking in a school crocodile with a ridiculous hat on; again, sloping disconsolately along in my early teens. When I take my son and my nieces to the pink sweetshop on the high street, there’s a little phantom with a wonky fringe standing next to them, looking on longingly. They’re terrible for your teeth, I tell her. You’ll regret it, when you’re my age.

And it isn’t just my own ghosts. I know (because my great-aunt told me) that there used to be a tram line right through the middle of Rochester High Street and so I see it, faint, shimmering at the edges. I know my grandfather must have walked along here, my grandmother, her parents, their parents, sisters, brothers. Connections cover the place like ivy, straggling across the shopfronts up to the roofs, sneaking, snaking over the pavements. Having spent the last few years working on Charles Dickens, I see him too, the eyes, hot and melancholy together, the natty waistcoat. He is walking, discoursing to a crowd of visitors who trail in his wake, hurrying to keep up. You’re quite right, Miss Havisham lives just up that road there, and Uncle Pumblechook’s shop is in this direction. Why yes, I did use this building in the Seven Poor Travellers. The gatehouse where John Jasper lives? We’ll pass it in a minute. I can show you exactly where Edwin Drood was standing when – And you’ll see the castle. I loved to sit there and read when I was a boy.

‘Of course the town had shrunk fearfully, since I was a child there,’ wrote Dickens, in 1860, in an essay which purports to relate his feelings on revisiting Medway for the first time in decades. He describes a sense of doubleness very similar to what I experience, of seeing the scene both as it is and as it was, the self-haunting. But you can’t rely too much on the details he provides. He conflates Rochester with Chatham, where he actually spent his boyhood, and brings a dead childhood friend back to life, purely in order to imagine her grown ‘fat’ and matronly. He had visited both towns regularly throughout the previous quarter of a century and at the time he penned this essay was living in the neighbourhood, at his house at Gad’s Hill. The dazed nostalgia is, mostly, bunkum. Dickens’ writing often tends towards disingenuousness when the subject is himself. He’s evasive. In this case it turns out he may have had an additional motive; concealing the fact that he had access not just to his own local memories, but to his mistress’s as well.

Rochester was home to Ellen Ternan’s uncle William, her father’s brother, who had married, in 1830, a locally-born woman called Catherine Haymen or Hyman and settled down to raise a nineteenth-century quiverful of children. And it was in Rochester that Ellen was born, in 1839. Soon after her birth her parents moved to the north of England to pursue a theatrical business venture which didn’t prosper. Her younger brother died; in 1846, after a period of illness, so did her father. Ellen’s mother embarked on an unsettled, peripatetic life, travelling all over the British Isles chasing theatre roles for herself and her daughters, though for Ellen, the youngest, these proved quite difficult to come by.

But in March 1850, according to newspaper reports, ‘Mrs and the Misses Ternan’ were at ‘the Rochester Theatre’ and in January of the following year they were there again; Mrs Ternan acting the main female role in a local amateur performance of Richelieu, a play by Edward Bulwer Lytton. The Ternan girls performed a dance ‘between the play and the farce’ and someone called ‘H. Haymen’ found favour in a comic role – this is probably Henry Haymen, who was a sort of cousin, their uncle William’s nephew by marriage. Two and half months on, when we get to the 1851 census, Ellen and her sister Maria seem still to be in Rochester. At least, their names appear as members of their uncle’s household on the High Street, as well as being listed alongside their mother and sister in St Clement Dane’s in London.

Did Ellen stay in Rochester? Did her uncle take charge of her, perhaps for a lengthy period? We struggle to trace her anywhere else in the early 1850s. Her uncle was reasonably prosperous, was married, had a number of children close to her age, so it looks to be at least a possibility. Her memories then, as a girl of 11 or 12, would belong to the city, would be circled by her cousins, and their cousins, who lived further along the High Street, their widowed uncle Haymen and his sons.

Ellen, Ellen as a resident of Rochester, adds a whole other layer of links and memories, opens up a new set of possible interpretations for the novels Dickens wrote during their relationship – Great Expectations, Our Mutual Friend, and _The Mystery of Edwin Drood_, the first and last of which are both set partly in Rochester, or somewhere that looks an awful lot like it.

Some readers think of the chilly Estella as a portrait of Ellen, with the flawed Pip, fatally ensnared by her, as Dickens. But given that Ellen was born locally – like Pip – and – like Pip – had an ‘uncle’ who lived on Rochester High Street (both a real one and a sort-of one in the Pumblechook mould, Mr Haymen), we might want to reassess. Mr Haymen’s wife died in 1847; she is buried in the precincts of Rochester Cathedral. Might she have inspired the dead Mrs Sapsea, whose widower lives on ‘the High-street’, and whose cathedral tomb seems to have been set to play an important role in _The Mystery of Edwin Drood_? It’s a question worth asking.

Perhaps even Our Mutual Friend should be rethought. The Haymens – Ellen’s aunt Catherine, Catherine’s brother – seem to have belonged to a family which had originally been Jewish. From the records of their baptisms we can identify their parents as Solomon Hyman and Keturah (or Rebecca, later Catherine) Patten, who married in London in 1799. Might Ellen’s affection for her technically Jewish relatives, the protection and warmth they had offered her, find an echo in the goodness of Mr Riah, and of the Jewish family who offer refuge and employment to the unprotected Lizzie Hexam? Presuming Ellen knew, why not?
See Ellen as a baby; see her walking along Rochester High Street with her cousins, clinging onto childhood by her fingertips, fearing what might come. See her mother in Rochester theatre, her sisters dancing. Ellen’s ghosts, everywhere, crowding her, when she returned in adulthood, which it seems very probable that she did.

The 1861 census records William Ternan living, widowed, but with some of his children, and a grandchild, in Strood. Strood lies across the river from Rochester, even closer to Dickens’ house at Gad’s Hill, not more than two miles distant. There is plenty of open country roundabout; easy enough for Ellen, visiting her uncle and cousins, to meet her lover secretly or as if by chance. See her there, striding towards him – no passive muse, she.
Only how she felt, seeing her own memories trapped inside the glass dome of a novel, we have no way of knowing.