Is this article weak, fair, good, or excellent?
How are you supposed to know? You haven’t even read it yet. But if I worked for TEGNA, I’d be getting that kind of feedback already, while I’m still typing away. It’s a built-in feature called the “story strength meter” that was recently integrated into the company’s proprietary content management system (CMS) called “TEGNA One.” “This is like when you’re creating a new password, and it tells you if it’s weak, good, or really strong,” says TEGNA’s chief digital officer, Adam Ostrow. “This does the same thing for stories. And it’s based on how well that story executes against our best practices.”
The new story strength tool is just part of an ambitious home-grown CMS that is still evolving nearly three years after launch — an example of collaborative and continuing innovation. As we mentioned in our widely read report on how to address local TV newsrooms’ recruitment crisis, making the workplace less onerous for overburdened producers is an important priority. But that’s by no means TEGNA One’s only goal. “I’d say a theme is integration and simplicity,” Ostrow says. “It’s also how to use technology to drive performance, as well as to power better content experiences.”
“Story strength is a live meter grading scale that you see at the top of any article that you’re writing in TEGNA One,” explains Jessica Mullins, a regional digital director based at Boise’s KTVB. “And as you’re writing it, you’ll be able to watch the ratings go from weak to fair to good to excellent. And there are built-in criteria that are weighted in different ways that will help you get your score up.”
The tool generates a score for your story on a 100-point scale, and it also offers drop-down menus with suggestions for how to improve your tally as you go along — adding video, say, or links, or more details. “Any story we do can benefit from being optimized,” says Ostrow. “We want the story to find an audience. So it’s really designed to help the creators do that.”
Ostrow, who joined TEGNA about four years ago after a decade at Mashable, realized that creating a new CMS would be a priority when a survey of stations revealed a satisfaction rate of only 30% with the previous system, which came from an outside vendor. Moreover, he says, “We were probably using five or six different tools to do all the things that we can now do in one.”
TEGNA used in-house product managers and developers, along with a New York City design firm (Code and Theory) to create TEGNA One — with a lot of input from employees in newsrooms around the country. “[We] asked for feedback on all the different core features as well as suggestions for things that the field would like to see to help them do their jobs better,” Ostrow says.
Regional digital directors like Mullins, who works with nine other stations as well as her own team in Boise, have a key role. “They’re our eyes and ears for the field,” Ostrow says. “So they play an important part in planning our product roadmap, as opposed to just the technology and product people.”
The result: features like the ability to send push alerts right from the CMS, rather than relying on an often sluggish third party as before; the ability to select video clips from station newscasts, all of which are accessible on the system; the ability to vet and integrate user-generated content that is uploaded directly into the CMS; the ability to share content easily with other stations, with rights management built in; and the ability to live-stream across OTT platforms, mobile apps, social media, and owned websites. “It does it all from one interface, and it’s ‘Click, click, click, here’s where I want it to go,’” Ostrow says. “And they can do that from inside TEGNA One.”
The most recent addition: headline testing. Users can submit up to four different headlines, which undergo real-time testing on the station’s home page. The algorithms select a winner based on proprietary engagement metrics. “It’s not just clicks, it’s kind of a special secret sauce,” says Mullins. “The best thing is that once there’s a determined winner, it will automatically change the headline of the article to the winning headline. So your digital producer, who’s already spinning ten plates — managing social media, the website, everything else, and writing stories — doesn’t have to remember to go back in a timely way and update their headline after that.”
A/B (or A/B/C/D) headline testing is not new, of course; it’s been standard practice in digital publishing for years. But integration into the CMS has made a dramatic difference at TEGNA. Ostrow estimates that when stations had to use an outside service for headline comparisons, they ran a couple of hundred tests a month. Now, thanks to the latest feature in TEGNA One, content creators ran 4,000 tests in September alone.
The story strength meter may sound a tad more Orwellian to you. It did to me at first. But both Ostrow and Mullins say that it’s used to drive better practices across teams and enhance quality, even though “it definitely creates a competitive dynamic,” Ostrow says. “Some people really want to make sure that their stories are good or excellent.” “We really emphasize to stations and producers that this isn’t about just getting a 100% score every time,” says Mullins. “It’s just this tool to help guide you to make the best possible content. It’s not about gaming the system.”
Ostrow says that thanks to the real-time story length tutorial, flagship Washington, D.C. station WUSA raised its average from a lackluster score in the 50’s to almost 80, “which is almost as high as you can go. And they’re also one of our top-performing stations in terms of growth this year, from a digital KPI [key performance indicator] perspective. So we do see correlation.”
The story strength feature does establish a new metric for evaluating the performance of stations and, when necessary, individual employees. Mullins says she doesn’t routinely look at individual producer scores, but “If I have someone where I’m monitoring performance, I can go back and look up the stories they’ve authored and see what their scores are. And that could be a constructive feedback conversation. But we don’t have any kind of structure in place where we’re holding everyone accountable to their individual scores.”
Ostrow says satisfaction with the new CMS is up to 85% in the latest field survey. The company still uses third parties for some functions, like Google for web analytics and outside ad tech. “We’re not trying to replicate the entire vendor ecosystem,” Ostrow says. “There are certain areas where we think it makes a lot of sense to own and control it.” For other broadcasters considering similar DIY projects, Ostrow advises thinking about “the bigger picture” — how to support them and keep improving them after they’re built.
“Prioritizing innovation is so important with the digital landscape and the future of news,” says Mullins. “We’re all figuring it out together. Making sure that your product is prioritized as a good experience and not a point of frustration, when you’re already combating the stress of the newsroom — it’s huge for morale.”
Get the Lab Report: The most important stories delivered to your inbox