As part of the SCOPE project's ongoing efforts to enhance research skills, the CSSM workshop series hosted a two-day event on web data acquisition from March 31 to April 1, 2025. Taught by Leon Fröhling and Dr. Jun Sun (GESIS), the workshop provided 12 participants with the fundamental skills of collecting and processing online data through a blend of theoretical instruction and hands-on practice.
The first day focused on working with Application Programming Interfaces (APIs). The sessions introduced how data is provided online and the central role APIs play in making it accessible. Participants learned to construct and test queries in the browser before moving on to programming API interactions in Python and exploring automation to scale their data collection efforts.
The second day shifted attention to web scraping. Following a broad overview of its principles and ethical considerations, attendees were introduced to the technical underpinnings of web pages. They then applied this knowledge to systematically extract content from static pages through a series of practical exercises.
The intensive workshop structure gave everyone the chance to experiment with the newly introduced tools, ensuring participants left with the confidence to apply these data collection techniques independently in their future research.