Test and Evaluation AI Expert

MANTECH seeks a motivated, career and customer-oriented Test and Evaluation (T&E) AI Expert to join our team in Washington, D.C. This is a hybrid position with one day per month onsite.

Responsibilities include but are not limited to:

• Provide Test and Evaluation expertise on a wide range of Science and Technology activities and missions to include test planning, test execution, data management and test reporting with emphasis on AI/ML enabled systems

• Provide T&E oversight support of multiple government programs and possess knowledge, expertise, and education in test and evaluation methods with emphasis on AI/ML enabled systems

• Provide T&E expertise for effectiveness and suitability in an operationally constrained environment

• Perform T&E Technical Analysis as part of an integrated program team providing technical advisory and lead system integrator technical oversight to the government test and evaluation division lead

• Evaluate the performance of Artificial Intelligence (AI) solutions using appropriate metrics and validation techniques. Conduct thorough testing to identify and address issues related to accuracy, robustness, and generalization of AI interfaces

• Design and develop machine learning algorithms, deep learning models, and statistical methods to solve complex problems; includes data preprocessing, feature engineering, model training, validation, and evaluation

• Provide insight to programming & implementation maturity; review of AI algorithms and models using programming languages such as Python, R, or TensorFlow

• Support data analysis & visualization reviews: Analyze risks with large datasets to extract meaningful insights and patterns. 

• Provide insight to model deployment & integration: Support the approach and analysis with deploy trained models into production environments, ensuring scalability, reliability, and performance. 

• Develop guides and training materials for T&E of Artificial Intelligence. Update and teach T&E of Artificial Intelligence seminar

Required Qualifications:

• Master’s degree in Computer Science, Engineering, Mathematics, Statistics, or a related field

• 15+ years of Test and Evaluation experience with a significant portion dedicated to AI testing and evaluation of complex systems

• Artificial intelligence and/or machine learning experience with T&E interfaces

• In-depth knowledge of machine learning algorithms, deep learning frameworks (e.g., TensorFlow, PyTorch), and statistical techniques

• Experience with data manipulation, analysis, and visualization libraries (e.g., NumPy, pandas, Matplotlib)

• Familiarity with software development best practices, version control systems (e.g., Git), and Agile methodologies

• Experience with natural language processing (NLP), computer vision, reinforcement learning, or other specialized areas of AI

• Knowledge of cloud computing platforms (e.g., AWS, Azure, Google Cloud) and containerization engine technologies (e.g., Docker, Kubernetes)

• Experience with big data processing frameworks (e.g., Spark, Hadoop) and distributed computing

Desired Qualifications:

• Defense Acquisition University (DAU) or DHS Level III Test and Evaluation Certification or equivalent

• Experience with DHS Acquisition Directives and Acquisition Lifecycle Framework (MD102/ALF)

• Knowledge, understanding and/or experience with the DHS (or DoD equivalent) Test and Evaluation policies, procedures and best practices for conducting, monitoring, and reviewing operational test and evaluation

Clearance Requirements:

• Must possess active Secret clearance

• Must be able to obtain DHS suitability

Physical Requirements:

• Must be able to remain in a stationary position 50%

• Frequently communicates with co-workers, management, and customers, which may involve delivering presentations. Must be able to exchange accurate information in these situations