Data Engineer

Data Engineer (Preferred Years of Experience - 8 years)

Level-Specific Responsibilities

Independently deliver analytics solutions and lead/manage assigned technical workstreams.

Partner with data engineering teams to design and implement data pipelines.

Build production-grade ML models and deploy them to cloud environments.

Develop reusable code libraries and modules for team use.

Liaise with clients, gather requirements, and translate into data science use cases.

Mentor junior team members.

Common technical skills:

Python: production-level proficiency; core: pandas, numpy, scikit-learn, statsmodels, scipy; application: streamlit (for dashboards), nltk (for NLP); SQL: Complex queries, joins, subqueries, window functions; Version Control: Git workflows (branching, merging, pull requests, code reviews); Software Engineering: developing modular, reusable code, OOP principles.

Stats & ML foundations: Linear and logistic regression, hypothesis testing, probability fundamentals; traditional ML: Ensemble methods (RF, XGBoost, LightGBM), dimensionality reduction (PCA, t-SNE); Model evaluation: Cross-validation, bias-variance tradeoff, regularization.

NLP: traditional: TF-IDF, word embeddings (Word2Vec, GloVe), topic modeling; articulate trade-offs: when and why to use traditional methods vs. transformer models.

Cross functional collaboration: experience partnering with data engineering teams, co-develop data pipelines/ETL processes, working with large (GB+ scale) datasets; experience partnering with DevOps for cloud deployments (AWS, Azure, GCP), MLOps.

Preferred: Generative AI/LLMs (BERT, SentenceTransformers, HuggingFace, OpenAI API), Computer Vision, Big Data tools (PySpark, Dask), Tableau/Data Visualization, experimental design, AWS Redshift experience, RAG architectures, fine-tuning language models, Time series forecasting: ARIMA, Prophet, seasonal decomposition