AI Data Engineer (contract)

AI Data Engineer (Contract)

Please note that this is a contract role providing services to Microsoft through external staffing partners of Allegis Global Solutions. If you are selected for this role, you will be employed by AGS and will not be an employee of Microsoft.

Summary

We are seeking a hands-on Data Pipeline Engineer to build and operate Python-based workflows that transform large document and text datasets into high-quality data for AI model training. This role sits within an applied science team that bridges research, innovation, and production to develop and train cutting-edge models.

The ideal candidate brings strong Python programming skills, proven experience processing datasets that exceed single-machine memory, and practical expertise managing data and processing workflows in Azure. Critically, candidates must have hands-on experience building datasets from the ground up — not simply working with pre-existing ones.

Job Responsibilities:

• Develop, maintain, and optimize Python pipelines for data ingestion, OCR processing, text extraction, cleaning, normalization, and deduplication
• Run and monitor large-scale batch processing jobs, investigate failures, and implement reliable retry and recovery mechanisms
• Build automated quality checks to identify missing or corrupted files, incomplete OCR outputs, duplicate content, formatting issues, and metadata inconsistencies
• Prepare training-ready datasets through filtering, text segmentation, tokenization, and conversion into agreed data formats
• Manage bulk uploads, downloads, and synchronization between local processing environments and Azure storage, verifying transfer completeness and data integrity
• Organize raw data, intermediate outputs, and released datasets with consistent naming, metadata, versioning, and source traceability
• Optimize processing throughput, memory usage, storage consumption, and cloud costs using batch, parallel, or distributed processing approaches
• Track dataset statistics, processing coverage, quality metrics, and exceptions, and produce clear operational reports
• Collaborate with applied scientists and model-training teams to translate data requirements into reproducible processing workflows
• Maintain tested, documented, version-controlled code and follow established data access, security, and governance requirements

Requirements:

• Minimum of five years of hands-on, data-related work experience
• Demonstrated experience in data engineering, data pipeline development, or a data science role with substantial production programming responsibilities
• Proven track record of creating original datasets, not solely consuming or transforming pre-existing ones
• Background in data science, mathematics, computer science, or a closely related field
• Strong proficiency in Python, including writing reusable modules, debugging, testing, logging, and handling processing errors
• Hands-on experience building datasets from the ground up, including original dataset creation and curation
• Experience processing large structured and unstructured datasets using memory-efficient, parallel, or distributed techniques
• Practical experience with Azure Blob Storage or Azure Data Lake Storage, including bulk data transfers, storage organization, and access troubleshooting
• Proficiency with Python data-processing libraries and formats such as pandas, NumPy, JSON/JSONL, CSV, and Parquet
• Working knowledge of SQL, Git, Linux environments, and command-line tools
• Strong troubleshooting skills, attention to data quality, and ability to communicate technical issues clearly
• Experience with OCR pipelines, scanned documents, multilingual text, or document-layout processing (preferred)
• Experience with Azure Databricks, Azure Machine Learning, Azure Data Factory, or workflow orchestration tools (preferred)
• Familiarity with Hugging Face Datasets, tokenization, near-duplicate detection, and training/evaluation data separation (preferred)
• Experience with dataset lineage, reproducibility, containerized environments, and CI/CD (preferred)

Additional Details:

• Location: Redmond, WA (Onsite)
• Duration: 8 months
• Pay Range*: $75.50 - $85.50 per hour
• Weekly Schedule: 40 hours
• Job Status: Non-Exempt
• Application Deadline: Apply within 72 hours of the posting date to ensure consideration.

Benefit plans are available to eligible employees, with employee contributions and plan costs varying based on the coverage options selected.

• Medical, dental & vision
• Hospital plans
• 401(k) Retirement Plan – Pre-tax and Roth post-tax contributions available
• Life Insurance (Company paid Basic Life and AD&D as well as voluntary Life & AD&D for the employee and dependents)
• Company paid short and long-term disability
• Health & Dependent Care Spending Accounts (HSA & DCFSA)
• Employee Assistance Program
• Time Off/Leave(PTO, Allegis Group Paid Family Leave, Parental Leave

Benefits are subject to change and may be subject to specific elections, plan, or program terms.

AGS is an Equal Opportunity Employer and will consider all applications without regard to race, sex, age, color, religion, national origin, veteran status, disability, sexual orientation, gender identity, genetic information or any characteristic protected by law.

If you require a reasonable accommodation related to the application or interview process due to a disability, please email accommodation@allegisglobalsolutions.com. This inbox is monitored solely for accommodation requests. For questions about open roles or to apply, please submit your application through the job posting, as this inbox is not monitored by recruiters and applications sent here will not be reviewed.

In accordance with the Immigration Reform and Control Act of 1986, employment is contingent upon verification of identity and authorization to work in the United States. All persons hired will be required to complete Form I-9 and provide acceptable documentation as required by law.

Please note that we may use artificial intelligence (AI) tools to screen, assess, or select applicants for this position. These tools may analyze application materials and assist our team in identifying candidates whose qualifications best match the requirements of the role. If you have questions about our use of AI in the hiring process, or would like more information, please contact us.

• We reserve the right to pay above or below the posted wage based on factors unrelated to protected classifications.

Individual compensation offered for this position within this range will depend on many factors, including qualifications, skills, relevant experience, job knowledge, geographic location, internal equity, and other pertinent job-related factors.