Guidance on Enhancing OSM Profile Pages with Contribution Data

[kcne]

Problem

We want to enhance the OSM profile pages with more engaging and informative features, similar to HDYC. These features aim to display user contribution summaries, activity patterns, and insights into changesets. Our goal is to integrate this functionality directly into the openstreetmap-website while ensuring efficiency and scalability.

Key Challenges:

1. Data Access:

   • The required data already exists in the OSM database, so we won’t rely on planet dumps for this integration.
   • For development purposes, we want to import a subset of data locally to test queries and ensure they work well with larger datasets.

2. Performance:

   • Querying user contribution data efficiently at scale is critical. Local testing will help us benchmark and optimize queries for production.

3. Integration:

   • The solution should align with OSM's infrastructure, avoiding external systems or unnecessary overhead.

---

Description

To achieve this, we’re seeking guidance on:

• Best practices for querying user contribution data within the existing OSM infrastructure.
• Efficient workflows or tools to import and work with a representative subset of data for development.
• How to benchmark and ensure scalability of queries before deployment.

---

Questions

1. Has anyone implemented similar features for OSM profile pages? If so, what approach did you take?
2. Are there recommended tools or workflows (e.g., Osmium, osm-history-renderer) for importing and testing user contribution data efficiently?
3. How can we test and benchmark queries locally without committing to a full-scale data import?
4. Can these queries be handled within OSM's infrastructure, or would a separate system be necessary for performance?

---

We welcome any advice, insights, or references to similar projects to guide us forward. Tagging: @gravitystorm @tomhughes @AntonKhorev @pa5cal

[pnorman]

• Changesets in the planet OSM files contain user data, but tools like Osmosis and Osmium cannot directly link users to contributions in a database.
• Full history dumps provide the necessary data but are massive (~223 GB compressed, ~3.7 TB uncompressed) and complex to process.

If you're looking in the context of openstreetmap-website, this data is already present in the database and you'd never look at the dumps. You'd only use the planet files if you were building something external.

[kcne]

Thank you for the quick response! Yes, we’re focused on the openstreetmap-website and plan to integrate this directly. Testing queries and performance locally with larger datasets ensures reliability and simplifies development. That’s why we aim to import a subset of data for development, enabling us to test and refine features effectively before integration.