Software Developer (Content Integration & Data Pipelines)

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Software Developer (Content Integration & Data Pipelines) based in Canada. We are seeking a skilled Software Developer to build and scale production-grade data pipelines that collect, process, enrich, and deliver online news and web content across a sophisticated media intelligence platform. In this role, you will help connect thousands of content sources to search, analytics, and AI-powered capabilities used by communications professionals. You will work across web scraping, content integration, backend development, and event-driven processing to ensure information is delivered accurately and efficiently. Using technologies such as TypeScript, Java, Kafka, and Solr, you will contribute to the architecture and evolution of high-volume content ingestion systems. You will play an active role in improving pipeline reliability, performance, observability, and data quality while following responsible web crawling practices. This remote opportunity offers the chance to solve complex engineering challenges, influence technical direction, and collaborate with a team focused on innovation, automation, and scalable data solutions. Accountabilities - Build scalable web scrapers and crawlers: Develop and maintain production-ready scraping and crawling solutions that collect online content responsibly, following standards such as robots. txt, rate limiting, and applicable terms of use. - Develop content integration services: Implement parsing, extraction, and normalization logic using TypeScript and JavaScript to transform content from diverse sources into consistent, usable data. - Design event-driven pipelines: Build and maintain Kafka-based services that process, enrich, and route content in near real time, supporting reliable and scalable data flows.

- Integrate external content sources: Connect third-party content providers, RSS and Atom feeds, and external APIs to expand and maintain content coverage. - Develop backend services: Design and implement Java-based RESTful services that support content processing, integration, and delivery across the platform. - Improve data quality and reliability: Establish and apply best practices for error handling, source management, validation, data normalization, and fault-tolerant pipeline operations. - Monitor and optimize pipeline performance: Implement observability and monitoring practices, investigate processing issues, and improve system performance, reliability, and operating costs. - Contribute to technical architecture: Participate in system design discussions, evaluate scalability trade-offs, and help define the technical direction of content acquisition and data processing systems. - Support search and indexing capabilities: Contribute to content indexing and retrieval workflows, collaborating with search technologies such as Solr to ensure content is structured and accessible. - Collaborate across the development team: Work with technical colleagues and other stakeholders to deliver maintainable solutions aligned with product and business needs. - Mentor and share knowledge: Support junior developers, contribute to code reviews and engineering best practices, and promote continuous learning within the team. Requirements: - Education and experience: A degree in Computer Science or a related field, or equivalent practical experience, with at least three years of professional software development experience. - TypeScript and JavaScript proficiency: Strong programming skills in TypeScript or JavaScript, particularly for content integration, parsing, and data processing. - Backend development experience: Solid experience building backend services with Java or a comparable programming language.