ResearchLanka AI — Research Analytics Platform for Sri Lanka

01The problem

Sri Lankan research output is scattered. A single paper can appear in OpenAlex, Crossref, SLJOL, and its author's university repository, each with slightly different metadata and no shared identifier. Anyone asking a national-level question — which institutions collaborate, how output changes over time, where a field is concentrated — first has to rebuild the dataset by hand.

02The pipeline

I implemented the harvesters for Sri Lankan university repositories. Most expose OAI-PMH, so that path handles XML parsing and resumption-token pagination to walk collections that are far larger than a single response; repositories without a working OAI endpoint are covered by a sitemap-based crawler instead.

The genuinely hard part is not fetching — it is agreement. The same author appears as different name orderings, initials, and transliterations; the same institution appears under its English name, its abbreviation, and a departmental sub-unit. Records are normalised and disambiguated on the way into PostgreSQL so downstream analytics count one entity once.

03The dashboard

On the query side I am building the React and TypeScript dashboard against a FastAPI backend: Plotly views for output and field-level trends, and Cytoscape.js graphs for co-authorship networks, where the interesting structure is between institutions rather than inside them.

The stack is containerised with Docker and ships to AWS via GitHub Actions, so the harvest and the dashboard deploy from the same pipeline.