I am doing my graduation internship at CBYTE. They had an interesting assignment: building a scraping platform. CBYTE needed TikTok data from specific profiles for a client. In the past, they tried building their own scraper using existing open-source code, but it kept breaking. CBYTE then switched to using Apify, but due to the high costs, they still wanted to develop their own scraping platform—one that would host multiple scrapers, remain stable, and use shared bypassing methods. Large companies often attempt to block bot traffic (and thus scrapers), after all. So it was up to me and a fellow student to build a solution for this.

We researched existing anti-scraping measures and techniques to bypass them. We then translated the results of that research into a functional scraping solution, where new sources could be added quickly and easily. We implemented techniques to automatically bypass anti-scraping mechanisms and developed helper methods to simplify the process of adding new sources.

The final scraping platform consists of three components:

• ScrapeUI (Nuxt): A web application, similar to Apify, where users can easily set up a ScrapeJob. For example, you can specify which TikTok usernames need to be scraped and later view the results of a completed scrape. The UI makes managing scrape tasks accessible—even for non-technical users.
• ScrapeAPI (.NET): A REST API that handles communication between ScrapeUI and ScrapeCore. This API manages the creation, launching, and tracking of scrape tasks. It also serves as a secure gateway to the collected data. By decoupling the API from the UI and ScrapeCore, the platform is built to be flexible and scalable.
• ScrapeCore (Python): The execution engine of the platform where the actual scraping takes place. ScrapeCore includes a framework for easily adding new scrapers. It also features built-in techniques to automatically detect and bypass anti-bot measures.

I find it a challenging and educational project. We had to research complex technologies and come up with a smart strategy to bypass anti-scraping measures—a constant game of evasion and detection with the ‘anti-scraping police’.

What I learned:

• How companies combat automated traffic (anti-scraping measures)
• Techniques to detect and bypass such blockades
• Effective communication with a client and translating their needs into technical solutions

I began my journey at Altura as an intern, tasked with an exciting project: scraping publicly available tender data from various sources in the UK and optimizing the Neo4j Graph database and it’s queries to manage the increased data load. I successfully completed the project in half the allocated time, allowing me to explore and learn additional skills, such as working on the .NET API and resolving complex bugs. Mutual satisfaction with my performance and the working atmosphere led to my continued employment at Altura after the completion of my internship. At present, I am responsible for ensuring the integrity of the scraped data and maintaining the scrapers.

What I have learned:

• Building and maintaining data scrapers
• Neo4j Graph Database
• Database performance
• .NET API

In 2021, I began my studies in Computer Science. Although I had some experience building websites, I couldn’t yet call myself advanced; I was still a true beginner. My learning journey started with SQL databases, progressing through object-oriented programming (Java and .NET) and modeling. After graduating cum laude from the introductory year, I delved deeper into algorithms and data structures, code design and design patterns, software engineering, and DevOps ( Docker, microservices). I also studied interaction design for crafting UI/UX. Moreover, I gained extensive knowledge in networking and security. My coursework included mobile development for both Android (Kotlin + Jetpack Compose) and iOS ( SwiftUI). Throughout my studies, I worked with various web technologies, including JavaScript frameworks like VueJS, React, and Angular, as well as back-end frameworks such as ASP.NET Core and Laravel.

Additionally, I completed several projects for real clients, where I learned to communicate effectively with both clients and fellow students. In 2023, I undertook an enriching internship at Altura, where I served as a .NET backend developer. I successfully completed my internship with a commendable mark of 9.8 out of 10 (because you can’t be perfect, you know 😉). This experience further honed my skills in backend development, solidifying my understanding of .NET technologies and best practices in software engineering.

What I have learned:

In short: a lot about algorithms and data structures, code architecture, databases, app/web technologies and project management

• SQL/NoSQL Databases: Design, management, and optimization of databases
• Object-Oriented Programming: Java, .NET, modeling and design principles
• APIs: auth, error handling, responses, documentation, etc.
• Algorithms and Data Structures
• Code Design and Design Patterns
• Software Engineering: development lifecycle and Agile methodologies
• DevOps: Docker, microservices architecture, CI/CD
• Interaction Design: UI/UX design principles
• Networking and Security
• Mobile Development: Android (Kotlin + Jetpack Compose) and iOS (SwiftUI)
• Web Technologies: JavaScript frameworks (VueJS, React, Angular)
• Back-end technologies: ASP.NET Core, Laravel, Node
• Professional Experience: Effective communication with clients and team members + Real-world project management and execution

I discovered my strong writing skills in Dutch and felt compelled to assist others with their texts. On Knoowy, clients approached me to proofread their theses, assignments, and various texts. This experience not only honed my ability to communicate effectively and market my services but also sharpened my meticulous attention to detail—akin to debugging texts for errors. Understanding the reader’s perspective became crucial for crafting clear and structured texts. This work significantly improved my own writing abilities and proficiency in documenting code. Providing feedback to clients post-proofreading also taught me the art of delivering constructive criticism.

What I’ve learned from this experience:

• Effective communication
• Selling services
• Clear writing (structure)
• Delivering constructive criticism

In rain or shine, from August 2018 until December 2024, I’ve been delivering mail and packages in the Netherlands. This experience has significantly boosted my discipline. Additionally, it has allowed me to improve my social skills: chatting with elderly folks, hearing the standard joke about disliking blue envelopes with bills (to which I replied I’d paint them a different color next time 😅), and experiencing various adventures along the way.

Delivering mail accurately to the right address is meticulous work. It has taught me to prioritize the customer’s needs, to be attentive, and to fully focus on my work. Of course, I’ve made mistakes at times, but I always made sure to personally make things right, further developing my sense of responsibility.

What I’ve learned from this experience:

• Social skills
• Customer-oriented thinking
• Precision in work
• Taking responsibility