03/11/2013 09:03 am ET

'Downton Abbey': Branson And Lady Mary Romance In Season 4?

On Season 4 of "Downton Abbey," there's a widower (one Mr. Tom Branson)" and a widow (Lady Mary). Natural soap opera logic dictates the two will have to get together at some point ... right? Wrong!

"Oooh, I can't do that," Michelle Dockery told TV Guide. "He's my brother-in-law!"

Dockery's Lady Mary became a single mother in the "Downton Abbey" Season 3 Christmas special after Matthew (Dan Stevens) died in a car accident. Allen Leech's Branson became a single father earlier in the season when Sybil (Jessica Brown Findlay), Lady Mary's sister, died due to complications from childbirth.

"While it would be great for drama, I don't think it would be right for those two," Leech told TV Guide . "As much as they respect each other, they come from very different worlds."

Season 4 production is already underway in the UK -- see the first photo of Lady Mary and her son here -- and Dockery said her character and Leech's Branson will be spending more time together ... just not in the bedroom.

However, there will be romance in the air. "Downton Abbey" is introducing a bevy of new characters to spice up Season 4, including two suitors for Lady Mary and a new nanny for Branson's daughter.

Lily James, cousin Rose, will join the cast as a series regular and Tom Cullen, Nigel Harman, Julian Ovenden will pop up as new characters. Cullen plays a possible suitor for Lady Mary, Lord Gillingham, a family friend of the Crawleys. Harman will play a new valet and Ovenden is set to appear as aristocrat Charles Blake. Dame Harriet Walter will appear as Lady Shackleton, a friend of the Dowager Countess, Joanna David is playing Duchess of Yeovil and Dame Kiri te Kanawa will play a guest who sings at the house. Look for Shirley MacLaine to return as well.

"'Downton Abbey' has seen many great characters visit the house over the years and we couldn't be more thrilled to welcome the new faces that will be joining the regular cast of 'Downton Abbey' in Season 4," executive producer Gareth Neame said in a statement. "We hope audiences will be as excited about them as we are."

"Downton Abbey" premieres in fall 2013 in the UK and winter 2014 in the US.

"Downton Abbey"