Toby Green is a Cofounder of Coherent Digital. Before Policy Commons initiative became a reality, it was a dream of Toby for over a decade. As Head of Publishing for Organisation for Economic Co-operation and Development (OECD), Toby led the development and launch of the organization’s online repository, iLibrary. After twenty years, Toby left OECD in 2019 to cofound Coherent Digital.
Every librarian he consulted spoke of the challenges of finding, cataloguing, and preserving policy grey literature. The production team began work, and Policy Commons launched in November 2020. Behind the scenes, Policy Commons employs multiple custom-built crawling technologies, AI tools, extraction micro-services, and a powerful search engine, all running on Coherent Digital’s proprietary Commons platform.
In this email interview for Open Interview, Toby shares his ideas and thoughts with Gopakumar V and Siddu, on collecting and facilitating grey literature for the scholarly community. He discusses on how grey literature is becoming a feature, rather than being a bug in scholarly communication and how the Policy Commons initiative has been active in different regions of the world in the grey literature domain. He also talks on why Coherent Digital is seeking relationships with Google Scholar, EBCSO and Clarivate and libraries to reach out to the mainstream users so they can discover a new world of informally published content.
.
• Gopakumar: What is your take on the importance of grey literature in the scholarly communication?
Toby: Grey literature is very important. If you look at the number of references to grey literature in scholarly journals and books, you will see how important it is, especially in the social sciences. I would also note that an increasing number of research institutes based in universities are publishing their findings as reports, not as journal articles or books. Outside the academy, research organisations (like NGOs and think tanks) routinely publish their findings as grey literature not as journal articles or books. There is also the question of whether preprints are a form of grey literature. If they are, well, then even in the hard sciences researchers are choosing to use a form of grey literature to rush out their findings. All this points to grey literature being very important in scholarly communication.
• Gopakumar: What inspired Coherent Digital to launch Policy Commons : Oceania, and why there is a focus on grey literature and that too this specific region?
Toby: The idea to build Policy Commons developed when I was working for the OECD . My brief was to maximise the dissemination of their reports and papers. Our biggest challenge was discovery. On the one hand, public search engines like Google presented challenges because they are optimised for consumers (because that’s where the advertising dollars are). This, as I’m sure you know, this makes it really hard to get onto the first page of their search results. On the other hand, specialist search engines like Web of Science, Google Scholar, and EBCSO were a problem because they do not index content from organisation like the OECD because they focus on indexing publications from mainstream publishers.
In speaking with my peers at the UN and other IGOs, I discovered we all had the same challenge. I then spoke with university and government librarians and discovered they recognised the problem only too well: they were spending hours helping their patrons track down our content. They wanted a dedicated discovery service that would make it easier to uncover hard-to-find reports, papers and other documents not just published by IGOs, but also by NGOs, think tanks, research centres, governments and cities. So, I left the OECD , co-founded Coherent Digital, and set about co-developing Policy Commons with the help of librarians around the world. We now index content from and drive traffic to the websites of 40,000 organisations – helping to boost the awareness of content that would otherwise remain very hard to find.
As for Oceania, the idea came from the success we had had in Canada with Canada Commons, a platform which gathers grey literature, primary source content and e-books from Canadian sources. We felt that Oceania, a region, like Canada, with a strong local identity, might want a dedicated platform where their knowledge, history and culture could be discovered and preserved.
• Gopakumar: What are the main objectives behind Policy Commons: Oceania? What problems does it aim to solve within the research and knowledge-sharing ecosystem of the Pacific Islands, Australia, and New Zealand?
Toby: As my previous answer implies, the aim is to create a single platform where knowledge from across Oceania can be discovered and preserved. Today, that knowledge is scattered across thousands of websites, making it time-consuming and hard to find. Worse, many are at risk because their funding is fragile. Our goal is to ‘layer’ a search engine across all these scattered websites so users can search across them all at the same time – and then to link users to the items they need. In parallel, we make copies of each item and put them in a dark archive. If the link to the original item breaks then we route our users to the archived. In this way the content is not only easier to find, access is guaranteed even if the original website has disappeared.
• Siddu: Are there any similar experiments by any other publishers of this scale? How different is your approach in facilitating the grey literature?
Toby: No, not to our knowledge. A small number of other services index policy content, but their coverage is tiny (maybe 500 sources) and about half the items they index are journal articles, not grey literature: most index in a year what we index in a week. We believe we are the only company indexing grey literature at scale for the larger benefit of the scholarly community. In the last year, we indexed over 500,000 items every month.
• Siddu: With over one million pieces of research, reports, and grey literature from over 2,000 organizations, how do you ensure the diversity and comprehensiveness of the content included in the database?
Toby: Diversity and comprehensiveness come from identifying a large, diverse range of organisations and other sources of grey literature. Once we’ve identified and checked out a source (to ensure their content is of sufficient quality and relevance for our collection) we automatically index the content on their website and include it in our database. From time-to-time, our tools re-visit their website to refresh our index, adding new content as needed.
• Gopakumar: How does the platform support the dissemination of critical research in the region, especially in communities where access to academic databases may be limited or fragmented?
Toby: In creating a one-stop discovery service, we make it much easier for librarians, researchers and students find content they need for their collections, research and studies. This will lead to greater dissemination and use of the content that’s sitting on websites that are often little-used because they are so hard to find by traditional search engines. This content is rarely included in other third-party services, like Google Scholar, again, because it is hard to find. That’s why we are building relationships with Google Scholar, EBCSO and Clarivate and can feed library catalogue systems so our services can be a ‘bridge’ over which mainstream users can pass to discover a new world of informally published content.
• Gopakumar: Given that content from smaller organizations is often “lost” or not prioritized by search engines, how does Policy Commons: Oceania address this and ensure these resources are easily discoverable?
Toby: We index all content in exactly the same way, we do not give a weighting to any item. We are also now using AI to create standardised summaries for every item – which is another way of levelling the playing field. This mean every item we index, whether from a famous, large organisation or an unknown, small institute, has an equal chance of discovery.
• Gopakumar: What role does technology play in the development of Policy Commons: Oceania, how do you envision this platform being used by researchers, policymakers, or local communities? Are there specific features designed to cater to these diverse audiences?
Toby: Technology plays a vital role, without it, there would be no way to index at the scale we do. And now, with AI, we can add value by doing things, like summary creation, that would impossible to do at scale and at an affordable cost. As for use, the platform is designed to make it easy for practitioners, researchers and students to find and work with the content we index.
• Siddu: What impact has already been achieved and how do you anticipate Policy Commons: Oceania will a greater impact on research and policy-making in the region, particularly in supporting more informed decision-making?
Toby: We are just starting, so it is too soon to measure our impact, but we hope it will become a go-to resource for anyone studying societal issues for Oceania right across the policy spectrum and its history and culture too. We already have some governmental bodies using Policy Commons, so hopefully the resources they find are already informing decision-making.
• Siddu: How do you plan to engage local organizations and communities in the ongoing curation and growth of the database to ensure it remains relevant and comprehensive?
Toby: We call our platforms ‘Commons’ because we want them to become resources that the community uses not just as a search engine, but also as a platform where they can upload and share resources they think will benefit the community. We are already open to co-creation, anyone can suggest new organisations for inclusion and we welcome feedback too. If we succeed in building an active user community, the resource will be both relevant and comprehensive.
• Siddu: What are your future plans for expanding or enhancing the platform? Will there be additional resources or features added in the coming years?
Toby: I think this depends on the previous question. If we succeed in building an active user community and, crucially, win the support and engagement of librarians from across the region, then, naturally, the resources and features that the community needs will be added to the platform. We have made a start and will continue to invest in leading development efforts, but what is key is that the community buys into to what we have started and works with us to co-create a resource that meets their needs.
* * * * *
Note · All the answers/ opinions expressed in this interview/document are of the interviewee. This blog post has CC-BY license.
Cite · Toby Green (24th April 2025). We are indexing grey literature at scale for the larger benefit of scholarly community (Blog post). Accessed on: https://openinterview.org/2025/04/24/toby-green-we-are-indexing-grey-literature-at-scale-for-the-larger-benefit-of-scholarly-community/
Credits · Assistance Santosh C Hulagabali; Introductory text & photo of Toby Green: https://coherentdigital.net/about-us/our-team/toby-green
Gopakumar V, PhD heads the Knowledge Centre and Library at Kerala University of Digital Sciences, Innovation and Technology (Digital University Kerala). He was head of the University Library of Goa University. He has served in different academic institutions and has rich experience of academic librarianship. He was instrumental in initiating UG and PG courses in library and information science at Goa University. He has been serving the library community as a trainer, speaker, author, research supervisor and organiser. He has a great interest in photography. Email: gopakumar.v@duk.ac.in
Sidlingappa M. Huded (fondly called as Siddu Huded) is an Assistant Librarian at National Law School of India University, Bengaluru. His areas of interest are: scholarly communication, archiving, open access and digital content management. He was part of Azim Premji University team who recently launched Schoolbooks Archive, an open-access international archive of schoolbooks and artefacts.
