Wilkie James and the 54th Massachusetts: From the carnage of Fort Wagner in 1863 to a life in Milwaukee

The five children of Henry James Sr. include some of America’s greatest thinkers. Henry’s oldest son, the philosopher and educator William James, is considered the father of American psychology. His second son and namesake Henry James Jr. wrote 22 novels, hundreds of short stories, and many volumes of biographies, travel writing, art criticism, and memoirs.

A third son, Robert, showed promise as an artist and writer until alcoholism derailed his career. Their sister, Alice, taught history but suffered from psychological and physical illness much of her life. Her sharply observed and insightful diaries, published after her death, are still widely read and admired.

Then there is the fourth son. Garth Wilkinson “Wilkie” James was born in New York City in 1845, Wilkie was an undistinguished student, experienced many failures, and died young and penniless.

Of all the James family siblings, guess who ended up in Milwaukee.

There is much more to Wilkie. He was an idealist. A war hero. A devoted friend to many, and a loving family man.

The five James children had an unusual upbringing. On inheriting a vast fortune, the patriarch of the clan devoted most of his time to intellectual pursuits. (His circle of friends included Ralph Waldo Emerson and Henry David Thoreau.) James provided an informal education for his children consisting of lessons taught by a succession of sometimes eccentric tutors, interrupted by frequent trips to the museums, art galleries, and cultural centers of Europe.

Shortly before the outbreak of the Civil War, Henry James Sr. enrolled his two youngest sons, Wilkie and Robert, in Franklyn Sanborn’s Concord Academy. Henry James Sr. approved of Sanborn’s educational ideas and admired his opposition to slavery. A close friend of radical abolitionist John Brown, the schoolmaster was one of six people having advance knowledge of Brown’s 1859 raid on Harper’s Ferry. The unsuccessful attempt to instigate a slave uprising ended in Brown’s capture by

Two of John Brown’s daughters were students at Concord Academy when Wilkie and Bob arrived in 1860.

A surviving letter describes Wilkie in his boarding school days, “Wilkie was incomparable: Besides being the best-dressed boy in school, and in manner and talk the most engaging, his good humor was inexhaustible.”

War broke out in 1861. Fired by the fervent anti-slavery climate of Concord Academy, both Wilkie and Bob enlisted in the Union Army.

“When I went to war I was a boy of 17 years of age, the son of parents devoted to the cause of the Union and the abolition of slavery,” Wilkie recalled in a speech to Union veterans years later. “I had been brought up in the belief that slavery was a monstrous wrong, its destruction worthy of a man’s best efforts, even unto the laying down of life.”

Wilkie enlisted in 1862, the same year the North reluctantly allowed Black Americans to join the military – on the condition they served under White officers. Both Wilkie and Bob volunteered as officers in Black regiments.

This did not make the boys universally popular. Racism was common on both sides of the Civil War. White officers assigned to lead Black regiments faced ridicule, even outright hatred from civilians and fellow soldiers. It was also a decision that carried added risk, for White officers of Black regiments were, by order of the Confederate Congress, to be executed on the spot if captured.

Wilkie’s regiment – the 54th Massachusetts Infantry – was one of the first formed. In his letters home, Wilkie protested the unfair treatment of the men under his command. Black soldiers earned $7 a month in cash while White soldiers received $13 – but added proudly, “They are cut out for soldiers in every way.”

On July 16, 1863, the 54th Massachusetts first encountered Confederate troops and succeeded in driving off a squadron of rebel cavalry after a sharp fight near Charleston, S.C. The clash was significant because the Black soldiers saved an all-White regiment, the 10th Connecticut, from near-certain annihilation.

A newspaper reporter of the time wrote, “The boys of the 10th Connecticut could not help loving the men who saved them from destruction … the dark-skinned heroes fought the good fight and covered with their own brave hearts the retreat of the brothers, sons, and fathers of Connecticut.”

Later, in a speech delivered in Milwaukee, Wilkie recalled, “It had become a living, breathing suspicion with us … that all White troops abhorred our presence in the army. Can you not readily share with me that indescribable sensation which a youthful soldier feels, who placed in a like situation, leading heroic negro soldiers on to victorious battle?”

Days later, in one of the inexplicable and bloody blunders that mark the actions of both sides in the Civil War, the 54th Massachusetts under the command of its Harvard-educated Colonel, Robert Gould Shaw, was ordered to lead the assault on Fort Wagner on Morris Island, a heavily defended rebel earthworks guarding Charleston harbor.

The regiment’s attack was the subject of the 1989 movie Glory. Historian Jane Maher describes the battle this way: “The initial attack on Fort Wagner was one of the worst debacles of the Civil War. It was poorly planned and even more poorly executed.”

The terrain funneled the advancing 54th into a narrow, concentrated mass of men who were easy targets for the fort’s massed rifles and artillery. It was more a slaughter than a battle. Col. Shaw was among the casualties.

In the aftermath, a Confederate officer wrote, “The dead and wounded were piled up in a ditch together, sometimes fifty in a heap, and they were strewn all over the plain for a distance of three-fourths of a mile … The negroes fought gallantly and were headed by as brave a colonel who ever lived.”

Wilkie, a few steps behind when Colonel Shaw died, recalled the battle in his Milwaukee speech.

“Gathering together a knot of men after the suspense of a few seconds, I waved my sword for a further charge toward the living line of fire above us. We had gone then some thirty yards, groping, but determinedly onwards … the ranks mowed down at almost every step. Suddenly a shell tore my side. In the frenzy of excitement, it seemed a painless visitation … A still further advance brought us to the second obstruction … in front of the ditch. The enemy’s fire did not abate for this crossing, and here it was I received my second wound, a canister ball in my foot.”

Canister rounds were one of the most feared weapons of the Civil War. As troops approached, artillery gunners would fire densely-packed charges pf .69-caliber lead shot, effectively turning the cannons in gigantic shotguns.

“As I stood faltering with the shock of this wound,” Wilkie said, “the advancing column, passing by me and over me, with deafening shouts and deafening curses, filled the alternating spaces of deathly missiles in the atmosphere.”

Badly injured, Wilkie managed to drag himself behind a ridge of sand. Here, two medical orderlies, retreating from the now-lost battle, loaded him on a stretcher. As they started carrying him to the rear, “A round shot blew off the head of one of the stretcher-bearers, producing a horrible and instant death,” Wilkie said. “We all fell down together, except the companion stretcher-bearer, who betook himself in a lively manner to the fastness of some secluded sand hole.”

The last thing Wilkie recalled before losing consciousness was a fellow soldier, his lower jaw shot completely away, who dragged himself to Wilkie and hovered over him with his blood pouring down on Wilkie’s face.

Wilkie woke up the next morning, three miles away, lying on the ground among hundreds of wounded and dying soldiers in an improvised field hospital, where a handful of medical professionals and civilian volunteers struggled to save who they could.

Harriet Tubman was among the volunteers at Morris Island. An escaped slave who returned south time and again to lead more than 300 slaves to freedom, Tubman served the Union Army as a cook, nurse, and spy. On June 2, 1863, she guided 150 soldiers and two gunboats on a route she had scouted up the Combahee River. The raid freed 727 slaves, burned several plantations, and captured Confederate supplies.

It was the first time in American history a woman – and a civilian woman at that – planned and carried out a military operation.

Tubman watched the assault at Fort Wagner. She later recalled, “And then we saw the lightning, and that was the guns; and then we heard the thunder, and that was the big guns; and then we heard the rain falling, and that was the blood falling; and when we came to get in the crops, it was dead men that we reaped.”

Clarissa “Clara” Harlowe Barton also served at Morris Island. A young patent office clerk in Washington, D.C. when war broke out, she was one of the Union’s first civilian volunteers. She organized donations of supplies, prepared food for the soldiers, and tended to the wounded – often rushing in while the fight was still raging. At the Battle of Antietam, a bullet passed between her arm and her body, tearing a hole in her sleeve and killing the soldier she was tending. She made a point of never patching the bullet hole in that garment. Another time a shell blew in the door of the room she in which she worked. She carried on as though nothing had happened.

Barton organized the field hospitals after the unsuccessful assault on Fort Wagner. After Union forces finally captured Morris Island, Barton wrote, “We have captured one fort – Gregg – and one charnel house – Wagner – and we have built one cemetery, Morris Island. The thousand little sand hills that in the pale moonlight are a thousand headstones, and the restless ocean waves that roll and breakup on the whitened beach sing an eternal requiem to the toll-worn gallant dead who sleep beside.”

Harriet Tubman spent most of her time carrying for the wounded. She vividly recalled the scene, “I would go to the hospital early every morning. I would get a big chunk of ice and put it in a basin, and fill it with water; then I would take a sponge and begin. The first man I would come to, I would thrash away the flies, and they would rise like bees around a hive. Then I would begin to bathe their wounds, and by the time I had bathed three or four, the heat would have melted the ice and made the water warm, and it would be was red as blood. Then I would go and get more ice, and by the time I got to the next ones, the flies would be around the first ones, black and thick as ever.”

In 2016, it was announced former slave Harriet Tubman’s image would eventually replace that of Andrew Jackson – himself a slaveholder – on the U.S. $20 bill.

Clara Barton went on to establish the American Red Cross.

Just one more of the endless wounded on the sands of Morris Island, Wilkie was oblivious to all this. Infection had set in and he probably would have died had it not been for a family friend, Cabot Russell, who came across Wilkie while searching the hospital for his own missing son. When his search proved unsuccessful (his son, he later learned, had died in the initial assault), Russell returned to Massachusetts with Wilkie in his care. On the voyage home, a naval surgeon finally removed the canister shot embedded in Wilkie’s shattered foot. The injury would cause him lifelong pain.

Wilkie had a prolonged and slow recovery. Although he could have resigned his commission honorably, he insisted on rejoining his troops in the field. After re-injuring his foot in a fall, and after another lengthy recovery, he was appointed aide de camp to General Gillmore and promoted to the rank of Captain.

When Charleston surrendered, in February 1865, Wilkie was there to witness the event. He was recognized by a reporter from the Boston Journal who wrote, “I saw a young officer, looking musingly and long toward Morris Island – sitting there, the old flag floating over his head, apparently unconscious of everything around him. He walked away at last – rather haltingly, for he was lame and wounded – still gazing toward [Fort] Wagner.”

After the surrender ceremony, Wilkie escorted the Confederate governor of South Carolina back home. As they rode together, the governor complained to Wilkie of the hardships Southern civilians were experiencing.

Wilkie wrote, “Just think of this immense slaveholder telling me … the pants he had on were the only ones he possessed.” The irrepressible Wilkie promptly offered to give his own pants to the governor, “which he refused, however.”

Wilkie and brother Bob suffered greatly in the war. Bob had nearly died of sunstroke while campaigning in Florida. The physically disabled Wilkie and his emotionally scarred brother had trouble finding jobs in Massachusetts when the war ended in 1865. Bob, through a family friend, eventually landed a position with a railroad in Iowa.

With Wilkie in need of an occupation, Henry James Sr. offered to bankroll the purchase of land in Florida and place Wilkie in charge. The idea was to establish a large farming operation to provide employment for newly freed slaves. It wasn’t entirely altruistic. The Jameses, like many Northerners of the time, snapped up cheap land in the devastated south in the expectation of vast profit.

But crops were poor. Wilkie knew little of farming and was unprepared for the hatred he encountered from the region’s long-time White residents. Finally, land values plummeted and the James’ family investment ended in ruin.

Bob, troubled and restless, had joined Wilkie in the Florida venture for a time, then moved on again, ending up in Milwaukee working for the Chicago, Milwaukee & St. Paul RR. Wilkie, at loose ends, joined his brother in Milwaukee and went to work for the same railroad.

In Milwaukee, both men fell in love with women from prominent local families. Bob married Mary Lucinda Holton, daughter of Milwaukee businessman Edward Holton, in 1872. Wilkie married Caroline “Carrie” Cary, daughter of early settler Joseph Cary, in 1873. They had two children: Joseph and Alice.

Wilkie quit his job and started his own business supplying railroad construction materials. He and a partner had close ties to influential people in the Milwaukee Road and the company’s success seemed guaranteed. However, a nationwide financial panic wiped out Wilkie’s fledgling company and with it his savings. He had even borrowed against his life insurance policy.

About this time, his novelist brother Henry James described Wilkie in a letter to a friend, “He is not particularly successful, as success is measured in this country; but he is always rotund and good-natured and delightful.”

He was also, although his brother was unaware of it at the time, increasingly ill. His wartime injury continued to trouble him and he seriously considered having his leg amputated. Other aches and pains developed and worsened, followed by a heart condition. Finally, a doctor diagnosed Bright’s disease, a then-untreatable and fatal kidney ailment.

Two months before his death, Wilkie wrote to his older brother William, “I am trying to gain strength if I can to bridge over the increasing demands on any vitality, This I do very slowly, having lost nearly all my appetite – so that it looks as if it would not be long before I shall peg out.”

Wilkie died on November 15, 1883. His passing did not go unnoticed. The New York Times ran an obituary, as did the Boston Evening Transcript.

The Milwaukee Sentinel devoted an entire column to Wilkie’s passing, “He possessed rare conversational powers, and was eminently social in his nature. He was a delightful companion, genial, unpretending, genuine. Everybody that knew him loved him, and his death will cause unaffected sorrow through the wide circle of his friends.”

Wilkie was 38 when he died. He left behind a wife and two children, aged 7 and 9, and considerable debt from his failed businesses. As his brother noted, Wilkie was not a success, as success is measured in this country.

But there are other ways to gauge a man’s worth.