Episode 48: Beyond The CMS #48 - How McGill Uses AI and Unified Teams to Manage Digital Transformation - Joyce Peralta (McGill University)

McGill University runs a thousand sites on its central platform, 500 more custom builds, and tracks 6 to 8,000 content edits a week. Joyce Peralta, Manager of Digital Communications at McGill, joins Chris Bryce on Beyond the CMS to talk about the content model project addressing all of it.

The number that anchors this conversation: 61 different terms used across the university for the exact same program component. Joyce's central point is that AI does not cause content inconsistency. AI amplifies inconsistency that already exists. The inaccuracy people see in AI answers comes from information presented differently across websites, not from information that is actually wrong.

McGill looked at building a separate content repository just for AI and turned it down. A second repository means a second governance and synchronization problem, not a solved one. Joyce points to Dries Buytaert's markdown experiment on his own blog, which found that AI bots still hit the website itself the majority of the time.

The content model work is federated across about 50 subject matter experts from departments across the university. Rollout runs concept by concept, and Joyce estimates the timeline at five years or more. Her real point on success: this cannot be a Communications project or an IT project. It has to be both working as one team, with alignment across the whole institution.

Key topics

• Why AI amplifies content inconsistency instead of causing it
• The 61 different terms McGill found for a single program component
• Why McGill rejected building a separate content repository for AI
• Dries Buytaert's markdown experiment and what it showed about AI bot behavior
• Federating a content model across 50 subject matter experts
• A concept by concept rollout Joyce estimates at five years or more
• Drupal, open source, and the Drupal EDU Slack group
• The student usability panel and how AI use among students is changing

Chapters

• 0:00 Welcome to Episode 48
• 1:05 Meet Joyce Peralta, McGill University
• 1:38 The velocity of change since Beyond the CMS launched
• 2:35 Inside McGill's web services team: comms and IT as one team
• 4:12 Who owns the university's content model
• 5:56 The vision behind McGill's content transformation
• 9:34 Taxonomy at scale: a thousand sites, 500 custom builds, 6 to 8,000 edits a week
• 14:33 Federated with 50 subject matter experts: the 61-term problem
• 16:19 AI amplifies inconsistency, it does not cause it
• 17:19 Why McGill rejected a separate AI-only content repository
• 19:10 How McGill actually uses AI in the content model project
• 21:35 Concept by concept: a rollout Joyce estimates at five years or more
• 24:02 Open source, Drupal, and the Drupal EDU Slack group
• 26:49 Three feedback groups, including the student usability panel
• 29:50 The human side of technology as AI reshapes expectations
• 31:32 Lessons for higher ed teams just starting this work
• 33:38 Where to find higher ed thought leadership
• 34:52 Wrap-up

Frequently asked questions

What is a content model, and why did McGill University build one? A content model defines the types of content an organization talks about and the terminology used for them. McGill built one because different departments were using up to 61 different terms for the same program component, which made the university's content difficult to navigate as a whole.

Why does AI make content inconsistency worse? AI cannot distinguish between different terms that mean the same thing across different websites. When one department calls something a specialization and another calls it a track, AI treats them as different concepts, which produces answers that look inaccurate even though the underlying information is correct.

Why did McGill decide not to build a separate content repository for AI? A second repository would multiply the governance and synchronization work instead of reducing it. Joyce Peralta cites Dries Buytaert's markdown experiment, which found that AI bots still access the actual website the majority of the time rather than a separate AI-only version.

How many websites does McGill University manage? McGill runs about a thousand sites on its central platform plus another 500 custom websites, with 6 to 8,000 content edits happening across the central platform every week.

How is McGill's content model project organized? The work is federated, built with input from about 50 subject matter experts across departments at McGill, rather than dictated by a single central team.

How long will McGill's content model rollout take? Joyce Peralta says the rollout runs concept by concept, department by department, and estimates the full transformation at five years or more.

Why does Joyce Peralta say communications and IT have to work as one team? No single department can define a content model solution for an entire university. Joyce says the biggest lesson she shares with other institutions is that communications and IT need a healthy, collaborative relationship from the start, and that institution-wide alignment is the critical success factor.

What platform does McGill University's website run on? McGill runs on Drupal, an open source CMS. Joyce credits the Drupal community, and specifically the Drupal EDU Slack group, as a major source of insight when McGill was building its governance model.

What is McGill's student usability panel? A feedback group of students, refreshed at the start of every school year, that McGill consults on topics including accessibility, new platforms, and how students are using AI to interact with university content. Because the panel runs every year, McGill can track how student AI usage is evolving over time.

How is McGill using AI in its own content model project? Joyce Peralta says her team used AI to process meeting notes, identify patterns in the data collected from subject matter experts, and reformat reports, not to generate the content model itself or suggest solutions.

What other feedback groups does McGill run besides the student panel? McGill also runs a web advisory committee that gives feedback on governance and ecosystem health, and a site manager and editor community of power users who meet monthly to discuss platform changes.

Where can higher ed teams find community and thought leadership on this kind of work? Joyce points to the Drupal EDU Slack group, Digital Collegium in the United States, PSC Web in Canada, and higher-ed-focused summits at conferences like DrupalCon.

Related from Dotfusion

We help enterprise and institutional teams build the content operations and governance behind large, decentralized websites. See our headless CMS services.

Links: Joyce Peralta on LinkedIn https://www.linkedin.com/in/joyceperalta | Dotfusion https://www.dotfusion.com

Full transcript

Edited lightly for readability.

Welcome to Episode 48

We're live. Welcome to everyone taking a moment out of their day to join us this Thursday, Eastern time, right at lunchtime. If you're joining from other parts of the world, or watching this in the future, we hope you get real value out of the conversation. If you're tuning in live, we can see questions posted on LinkedIn and we'll do our best to get to them. If you're watching this later on YouTube, we monitor the comments there too and we're always happy to respond.

The whole idea behind this show came from listening to Car Talk on NPR: two guys who could talk through anything under the hood in a completely down to earth way. We wanted that same kind of conversation about content platforms, and the future of content itself, with thought leaders in the industry.

Meet Joyce Peralta, McGill University

And we have one of the great thought leaders in the industry joining us today, Joyce Peralta. She's the Manager of Digital Communications at McGill University in Montreal. Thanks for joining.

Thanks Chris. I'm really happy to be here today and excited to have this conversation.

The velocity of change since Beyond the CMS launched

We were just chatting before we went live, and your comment about the velocity at which things are changing struck me, because it connects to why we named this show the way we did. Three years ago, when we started Beyond the CMS, it was a bit more about looking past the platforms themselves, into what the future of content would look like. You had a great starting point earlier, that this isn't so much about the platforms or the tech, it's really about the content. Maybe a good way to start is to ask you to describe a day in your life, and some of the projects you're working on.

Inside McGill's web services team: comms and IT as one team

Fantastic. Our team at McGill, our web services team, is actually two teams working together. Half of us are in IT, the developers, and the other half, the team I sit on, is communications. Together we provide a platform for people to build websites at McGill, with a central template and the training and resources attached to it, plus all of the support required to help people build great websites using that platform. We write articles, we run community events, we provide consultation support, and we define the standards and governance for our digital spaces. We work with people from across the university, because we aren't necessarily the ones who come up with information security rules, accessibility requirements, or brand usage guidelines. We bring the people who own those best practices together, and we produce resources for how those best practices need to play out on our websites. That's a summary more than a day to day, but hopefully it answers the question.

Who owns the university's content model

So is it safe to say that, to some degree, you're responsible for the content and the model supporting that content for the entire university?

Yes. Well, our web services team is. And I think that's an interesting distinction to make, because when you're talking to me, I'm in communications, but IT is also a very important part of this equation, and we talk about ourselves as a unified team. So yes, we're responsible for providing the platform and all the resources needed to create and deliver great content. But it's important to say that doesn't sit in one team, and it isn't limited to our teams either. I think the real strength of our governance and our processes is that we produce our resources for governance and content creation in a federated manner, with input from departments across the university. We'll work with our equity team, our translation team, and others. This isn't a sense of one department dictating how things should be done. It's really about bringing all of the different subject matter experts across the university together, so we're collaboratively producing the resources, guides, and tools needed to create great experiences.

The vision behind McGill's content transformation

In our pre-show chat, you mentioned that other universities and higher ed institutions reach out and wonder how you're doing all of this. Something that strikes me as smart and important is this collective nature of comms, IT, and extended teams overall. Can you share your vision for the migration, the transformation?

Sure. A lot of people are talking about transformation these days. When we started down this path, there was a lot of uncertainty about what it was going to look like. But I feel like we had a good base to start with, because we'd worked hard to produce foundational resources and governance that had buy-in from across the organization. Because we had processes in place for working with our community on our best practices, when we started talking about this content model project and a possible restructuring of our platform, it was easier to get buy-in, because people trusted that we would involve voices from across the community and produce a plan that considered input from different departments.

In terms of where the transformation is going, we're real fans of doing things in an iterative way, so there's uncertainty in exactly where things will end up. If somebody tells you they know exactly how this is going to turn out, you have to ask yourself if they're trying to sell you something, or if they're being realistic. We have a sense of the problems we need to address right now, but how things play out eventually, there's some uncertainty there, because the way people use our platforms and interact with content is evolving. What I can say is that we recognize we need very structured content production workflows, a very structured way of storing our content, and platforms that let us deliver content in multiple ways. That's where we're at: putting a plan in place to make that content transformation happen.

Taxonomy at scale: a thousand sites, 500 custom builds, 6 to 8,000 edits a week

Taking a step back, because some folks listening aren't from higher ed: can you talk about the depth and breadth of what you're dealing with from a taxonomy perspective? I've wandered around higher ed websites myself, and it's not obvious how you even organize a user journey across all of that.

It's true, these are very large environments from a content scale perspective, and historically it's been managed in a decentralized way. That's partly because in higher education we value independent thought and autonomy, so historically our departments have had the ability to manage their own content. That's meant that even when a department followed a great practice of doing user research and designing a solution specific to them, they've produced something that likely isn't similar to what any other department has. So what we have is hundreds, maybe thousands, of different content models across the organization. We have a thousand websites on our central platform at McGill, another 500 custom websites, and internal platforms and services running in the background, some of which interface with our public websites. All of those have individual content models and content structures, and right now there's a lack of connection between them. What that means is when people try to navigate our content as a whole, which is how people actually interact with it, they don't stay on one website. They go from website to website to complete their tasks, and every time they land on a different one, they're faced with a different content structure, and sometimes with information that contradicts the last website they were on.

How do you keep your hands wrapped around everything?

That gives you a picture of the situation. We also have all of those websites and platforms constantly being updated. On our central public web platform alone, we're tracking 6 to 8,000 edits a week. The content is constantly evolving. There's a lot to consider in making this transformation, and part of it is recognizing this happened because we never had a content structure for the university in the first place, which left people to follow their own best practices when building their own web solutions. This is nobody's fault, maybe it's ours, since we're the team that provides the central resources, but we'd never had the conversation about what our domain model should look like, what domains we talk about, or how information should be structured. We'd never had that conversation before.

Federated with 50 subject matter experts: the 61-term problem

I'm making some assumptions as I try to understand the different properties, but an example might be a website for a particular department, and Department A might have a fundamentally different mandate for communicating than Department B. It's almost like there's a calibration you're architecting, that acknowledges keeping things open, yet having a model, which is a little contradictory at the same time. It looks like you're trying to find that balance through open dialogue and feedback from the communities of property owners and users.

That's exactly it. As we've worked on this content model project, we've talked with different departments to identify the types of content we talk about, and the terminology used for that content. As part of the project, we talked with about 50 subject matter experts across the university about how they structure content on their websites. Two of the types of content we talk about a lot, of course, are programs and courses. But one of the surprises: for program components, we counted 61 different terms that people use to describe them across their websites. On one website it might be specialization, on another it might be track, on another it might be option or area of focus. So as a prospective student goes from website to website trying to figure out what program they want to apply for, they have to figure out every time what that site is calling program components, and that's what makes it really difficult for people to navigate our content as a whole.

AI amplifies inconsistency, it does not cause it

And we're seeing now with AI that it's really amplifying those inconsistencies, because AI also isn't able to distinguish between these terms being referred to differently on different websites. It's producing inaccurate results. An important distinction there: it's not producing inaccurate results because the information is inaccurate, it's because it's inconsistently presented on different websites, and that's what's causing the problem.

Right. I was thinking, if you extend that journey beyond your own properties, what if I'm looking at three different universities, all of which are using confusing synonyms? What do you actually do? It's almost like there should be a global head of higher ed models, a higher ed library everyone can use.

I agree, for content structure, and I think design could be interesting too, a consistent design system across universities.

Why McGill rejected a separate AI-only content repository

How are you factoring in producing content for the robots? Is that part of your model, under the presentation layer as well, and does that give you a whole other container of things to manage?

In the rollout of our content model, we're contemplating different possibilities for aligning our content with the model, and one of the things we considered was producing a separate repository of data just for AI. We've dismissed that, because we recognized it would just elevate the governance and synchronization issues we already have. I saw Dries Buytaert talk on his blog recently about how he created a markdown version of his website for AI, to see the difference in how AI crawled it versus the separate version of his content. The results were interesting: the website itself was still, the majority of the time, the version that was accessed by the AI bots.

So still prioritizing the human there.

Yeah, exactly, which we should do anyway. These are problems we should have addressed a long time ago. It's interesting that it's AI now that's making us have this elevated interest in addressing it, but I'm happy we're going to address it at all, because we really should have done this a while ago.

How McGill actually uses AI in the content model project

Are you able to share how your team is leveraging AI to help grasp this huge amount of content you have? Are you using it to suggest thematic content models, or how does a team even sort through all that information to come up with an appropriate set of models that will work across an organization of your size?

In the content model project, I used AI a lot, but I didn't use it to generate ideas or provide suggestions. I used it more to help process the notes we were taking at meetings, identify patterns in the information and data we were collecting, and reformat reports and things like that. It wasn't so much using AI to help identify the problems, we talked to people to identify those problems, but once we collected that data, we used AI a lot to sort out the information and present it in a way we could analyze, to determine what a good solution would be. We're considering using some kind of semantic search replacement tool to update information across our websites, but there are a lot of questions to ask, because we ultimately don't want to create a second repository of information, so we'll still have to update our websites. There are also a lot of change management questions, working with departments to adjust their workflows to incorporate a semantic search solution, and whether that's something people will accept. And there's the question of whether AI in this kind of solution will be able to draw correct conclusions across our whole content body.

Concept by concept: a rollout Joyce estimates at five years or more

So we're thinking, if we're going to do this in the least risky, most thorough way, it'll be more of a concept by concept approach, working with departments to restructure content one concept at a time, and considering all the related changes that need to happen alongside it: what governance needs to be in place, what our workflows should look like, what changes we need to make to our infrastructure for people creating content, content being stored, and content being delivered across the organization. That's going to be a very long, drawn out rollout. I was speaking with our senior developer about it, and we were thinking that approach could take five years or more. In presenting options to our stakeholders next week, we're going to talk about what to prioritize: doing this thoroughly, considering change management and all of the governance, infrastructure, and capacity changes, or doing it in a more accelerated way, which carries more risk and more challenges. Or they could expand the capacity of our teams, that's an option too, get more people on board so we can roll this out faster.

Open source, Drupal, and the Drupal EDU Slack group

I was listening to you speak a few weeks ago in Montreal, and what stuck with me was the pattern of openness you and your organization are committed to. We haven't talked much on this show about open source software. Do you want to chat about your stack and infrastructure, the pros and cons, and some of the things you've discovered along the way?

I can talk about it from our content perspective, since I'm not on the development team, though I used to be. In general, we use Drupal. In this process, not just this content transformation, but even in the past when we were laying the foundations and adjusting our governance, I was constantly interacting with the Drupal community to get insights from other people's perspectives, and bringing a lot of those lessons back to our university. When we first started our governance revisions, I sent surveys out across higher education to numerous universities to get input on what they were doing, and the majority of responses came from the Drupal EDU Slack group, people in the Drupal community sharing insights on their governance and how it was reflected in their platforms. That's one of the important things about open source, it allows us to move forward more quickly on initiatives and leverage the benefits of the work other people are doing. On another scale, we have a very engaged community of practice at our university, and I think part of that is being on an open source platform that lets us have conversations about our resources, platforms, and tools in an open way. There are so many strengths to being on an open source platform, and I'm glad our institution is part of an open source community.

Three feedback groups, including the student usability panel

You mentioned earlier that one of the great things you do is recruit users and members of different communities who might be experiencing your sites, and they give you amazing feedback. What's some of the interesting feedback you've gotten from those groups?

We have three main feedback groups. We have our web advisory committee, which a lot of universities have, and they give feedback on our governance and initiatives to improve the health of our ecosystem. We have our site manager and editor community, power users across the units who meet with us monthly to talk about platform changes and their thoughts on improvements. And then the one I really love is our student usability panel. We're in the process of recruiting new students for it now, we do that at the start of every school year as the previous students graduate. They're a group that reflects the diversity in our community, different audiences, different programs, different perspectives. We bring the students together and have conversations about everything from their use of AI, to our new course catalog platform, to our room booking platform, to accessibility and inclusion in our digital spaces. There are a lot of really interesting insights, and in particular, it's an evolving conversation in terms of how people are interacting with our content these days. We've been asking specifically about AI use in our student population, and over the years we've had this conversation, we can see the evolution of how student usage has changed, and is changing. It's a really valuable perspective for us to experience, and I'm grateful, they're such a great group of students. It's one of my favorite things about my job.

The human side of technology as AI reshapes expectations

You also mentioned something I hadn't thought about before, that there might be a growing movement against technology in general, with AI pushing that out a bit. How's that impacting how you're architecting the future?

I think how that will play out in our spaces, well, it doesn't really remain to be seen, because a big focus of everything we do in creating our governance, in creating these experiences with our community, in having these feedback mechanisms, is that we really try to highlight the importance of the human side of our technology. I think maybe it won't be something that changes, but it's something that will continue: we'll keep focusing on the human side of our technology and what drives the way we evolve our platforms. With this conversation about our content model and our need to structure content, at the root of it is an interest in making it easier for people to access our content. We're not doing this for AI, or to implement new AI-powered tools. We're doing this so people can access our content more easily, whatever that means. We're having these conversations and making these changes, and they're starting with people, and discussions with people.

Lessons for higher ed teams just starting this work

Time just whizzed by. I've got a whole bunch more to ask you, but maybe as a parting bit of discussion, we hope to attract folks from higher ed listening to this in the future. Is there a big lesson learned recently that you'd want to share with people looking to take on a project similar to yours, who are just at the foundational stages?

A big one I'm hearing about from other universities that contact us is the importance of communications and IT working together. If you're going to do this project, there's no single department that can define this solution for the whole university, and it especially can't be a Communications thing or an IT thing on its own. It needs to be IT and Communications together. If you don't currently have a healthy, collaborative relationship between Communications and IT at your institution, that's a really important place to start. Another key aspect is having institution-wide alignment around the goal of the work you're doing, recognizing that it can't be one department dictating a solution for the entire university. You need to bring the entire university together under a single vision for what you're trying to do, because historically it's really hard in higher education to roll out any initiative effectively without institution-wide alignment. That's going to be a critical success factor.

Where to find higher ed thought leadership

You mentioned that other universities reach out to you, and I'm assuming you reach out to them too. Is there a higher ed portal where this kind of information gets exchanged? How does someone tap into the greater thought leadership of this modernization work in higher ed?

The Drupal EDU Slack group is great, if you're on Drupal. There's also Digital Collegium in the United States, a community for marketing and communications folks in higher education, and PSC Web in Canada, a related organization. There are a lot of professional higher education groups you can get involved in. When I go to broader conferences like DrupalCon or ConFoo, they'll often have summits focused specifically on higher education, and those are helpful discussions to be part of.

Wrap-up

Well, we're going to wrap things up. Thanks Joyce, really great hearing your thoughts. If folks are listening and want to reach out to you, LinkedIn?

I'm on LinkedIn, yeah. Always happy to connect.

Thanks for your time, and thanks to anyone listening now or in the future. We'll be back some time in the next few weeks with episode 49 of Beyond the CMS. Have a wonderful day.