TV Watch
By Lesley Messer and Stephen M. Silverman
01/28/2013 at 01:20 PM EST
Downton Abbey season 3 cast
Carnival Film & Television/PBS
How seriously do TV viewers take what happens to the Crawleys and those in their sphere on
Downton Abbey?
So seriously that, as the series' creator and writer Julian Fellowes recently admitted to PEOPLE, he even gets yelled at "a bit" by members of the public when something they dislike takes place on the show.
"And they know it all," the Oscar- and Emmy-winner, 63, said about the public's reaction. "They have opinions that are better than mine."
SPOILER ALERT: Details of Sunday's broadcast are about to be revealed.
Sunday, just as the cast was named TV's best ensemble at the 19th annual Screen Actors Guild Awards, viewers got a tremendous jolt certain to elicit a potent response: Lady Sybil (Jessica Brown Findlay) died as a result of childbirth.
The series' executive producer, Gareth Neame, 45, is trying to see this upheaval as some kind of an advantage.
"These exits can be good opportunities for the drama," he told PEOPLE, "because they allow you to take the story in a different direction and change things in a way you hadn't quite thought of.
And while admitting, "it can be seen as a blow when you lose characters," Neame also said, "it can very often be as much an opportunity as it is a loss."
Fellowes, meanwhile, believes that the public is sophisticated enough to know "that when Sybil dies or when [another character, whose identity is being concealed] dies, it is because the actor wanted to leave [the series]" – rather than anyone's dying just for the sake of the plot.
"So, however sad it is, and we're all sad, it isn't a question of anybody being killed off in that way," the writer insists. "They would both be in the series till the end of it, if it was up to us. "
As for argumentative viewers, he says, "They'll say, it's such a shame, it's such a shame ... but you don't so much get shouted out for being horrible."
With Sybil now departed, the dynamic of the sisters – Lady Mary (Michelle Dockery) and Lady Edith (Laura Carmichael) – is bound to change, both on screen and off.
"The three girls became very, very friendly," said Fellowes, who also said that season 4 will, like the current one, take place amid the social upheavals of the 1920s. "They were sort of like sisters, really, because they were always going about in each other's trailers and gossiping and all that stuff. I think they wanted to be like a group of sisters and they made it happen.
As it is, he said, the formidable cast makes the set is a "jolly" one, "and Maggie [Smith]'s sort of den mother, and I suppose Hugh [Bonneville, who plays Robert Crawley, Earl of Grantham] is sort of the father of the show, but they all get on. Of course, it's lovely for the young ones to be in such a successful show, but the older ones have had a pretty good share of success, anyway."
• Additional Reporting By SIMON PERRY
