Available for opportunities

Hi, I'm Arslan Shahid |

Senior Data Engineer, AI/ML Solutions Engineer & Data Architect with 7+ years of expertise in Big Data, AI/ML Platforms, Data Governance, Cloud Platforms, and Enterprise Data Architectures across Government, Telecom, Banking & Healthcare sectors in the UAE & beyond.

0+ Years Experience
0+ Projects Delivered
0+ TB Data Managed
0 Certifications
0-node Clusters Deployed

Turning Data Complexity Into Business Value

Big Data
AI/ML
Data Governance
Cloudera CDP

I am a Senior Data Engineer, AI/ML Solutions Engineer, and Data Architect based in Dubai, UAE, with 7+ years of hands-on experience leading enterprise-scale data migrations, big data platform management, AI/ML deployments, data quality, and data governance initiatives across government, healthcare, banking, telecom, and transport sectors.

At Emircom LLC, I lead data engineering and AI initiatives for RTA Dubai — deploying production ML models on Dataiku DSS, building GenAI/RAG chatbots, developing parking demand forecasting pipelines, and architecting scalable ETL/ELT on Informatica BDM and Cloudera CDP. Previously at Hypernym FZ LLC, I managed a 700+ TB enterprise data platform as Technical Lead for RTA Dubai across ETL, BI, and infrastructure.

I bring deep expertise across the Hadoop ecosystem, Informatica suite (BDM, IDQ, EDC, IDMC), Dataiku DSS, Microsoft Azure, Databricks, Snowflake, and modern governance frameworks including NDMO, GDPR, and HIPAA. I am passionate about building data platforms that are reliable, intelligent, and impactful at scale.

Dubai, UAE
Sr. Resident Engineer, Emircom LLC
BS Computer Science, PMAS University
arslanshahid1997@gmail.com
Let's Connect

Technologies & Expertise

Big Data & Hadoop

Cloudera CDP/CDH Hadoop / HDFS Apache Spark PySpark Hive / HiveQL Impala Apache Kafka Apache Nifi YARN HDP / HDF

ETL & Integration

Informatica BDM Informatica IDQ Informatica EDC Informatica IDMC Informatica Axon Azure Data Factory Oracle GoldenGate Apache Sqoop SSIS Alteryx Talend

Cloud & Modern Stack

Microsoft Azure Databricks Snowflake Azure Synapse Delta Lake Kubernetes Elasticsearch Cassandra

Data Governance & Security

Data Governance Data Quality Metadata Management Kerberos Apache Ranger NDMO Framework GDPR / HIPAA Data Lineage

BI & Analytics

Power BI Microstrategy Vertica Oracle Exalytics Pentaho BI

Programming & OS

Python Scala / Java Shell Scripting SQL / HiveQL Linux / RHEL R TensorFlow

AI & Machine Learning

Dataiku DSS ML Lifecycle Management Generative AI RAG (Retrieval-Augmented Generation) Large Language Models ML Pipeline Orchestration TensorFlow Predictive Modelling
Expert Advanced Intermediate

Professional Experience

Sr. Resident Engineer

Emircom LLC
Dec 2025 – Present Dubai, UAE
  • Leading Data Engineering and AI/ML initiatives for high-profile UAE Government client (RTA Dubai), ensuring secure, scalable, and compliant data architecture
  • Deploying and managing production AI/ML models on Dataiku DSS, including a GenAI/RAG-powered enterprise chatbot and a parking demand forecasting model
  • Architecting, developing, and optimizing high-performance ETL/ELT pipelines using Informatica BDM and Cloudera Data Platform (CDP)
  • Designing distributed data workflows leveraging Hadoop ecosystem (HDFS, Hive, Impala, Spark) within CDP for large-scale transformation
  • Provisioning and governing structured and unstructured datasets for machine learning — managing full ML lifecycle from data prep to model deployment
  • Establishing validation frameworks, data profiling, reconciliation checks, and monitoring mechanisms for accuracy and audit readiness
Dataiku DSSGenAI / RAGInformatica BDMCloudera CDPSpark

Senior Big Data Engineer

Hypernym FZ LLC
Mar 2022 – Nov 2025 Dubai, UAE
  • Managed entire Enterprise Big Data Platform with 700+ TB of data as Technical Lead of ETL, BI, and Infrastructure for RTA Dubai
  • Designed PoC involving Snowflake as primary data warehouse for an AI-powered agent platform, including data ingestion pipelines and model integration
  • Seamlessly integrated structured and unstructured data into Databricks via Informatica IDMC; managed Delta Lake bronze-silver-gold architecture
  • Migrated Microsoft SSIS ETL workflows to Informatica BDM for improved scalability and maintainability
  • Enforced Data Governance & Cataloging (Informatica IDQ, EDC, Axon) to ensure compliance, visibility, and secure access to sensitive data
  • Successfully upgraded all environments (DR, PROD, Staging, Dev) from CDH 6.1.1 to CDP 7.1.9, eliminating legacy tech debt
SnowflakeDatabricksInformatica IDMCCDPData Governance

Big Data Engineer

Metis PVT LTD
Jun 2019 – Feb 2022 Islamabad, Pakistan
  • Led Big Data & Analytics Department, overseeing ETL, BI, and infrastructure operations across 10+ concurrent projects
  • Designed and deployed 45-node Cloudera (CDH) cluster for Telenor Pakistan — achieved 75% increase in data processing speed and 35% overall system performance improvement
  • Managed 45-node Cloudera (CDH & CDP) cluster operations including Kerberos security, Kafka, and Vertica 9.2 → 12.0.4 upgrade
  • Led deployment of 16+ node Hortonworks HDP cluster for Jazz (Lahore) with Cassandra/NiFi-based fraud management; deployed 8+ node HDP cluster for Islamabad Police national security project
  • Developed distributed data processing pipelines using Spark (Scala and Python) — improved Spark job performance by 28% through optimization
  • Implemented Oracle GoldenGate CDC for DWH-to-Hadoop migration at PTCL on Oracle Big Data Appliance
HadoopSpark ScalaKerberosCassandraOracle GoldenGate

Key Projects

Government UAE

Enterprise Big Data Platform

Roads & Transport Authority (RTA) Dubai

Managing enterprise Cloudera CDP (7.1.9) across 4 clusters (Dev, Staging, Production, DR) handling 700+ TB data from 50+ data sources. Full data governance, pipeline management, and AI dataset preparation.

Cloudera CDPInformatica BDMHiveImpalaKafkaPowerBIMicrostrategy
Telecom Pakistan

CDR Data Lake — 45 Node Cluster

Telenor Pakistan

Designed and deployed a 45-node CDH & CDS Cloudera cluster. Built CDR data lake with Kafka brokers, implemented enterprise security (Kerberos/Sentry), Vertica upgrade (9.2 → 12.0.4). Achieved 75% increase in data processing speed.

Cloudera CDHKafkaKerberosVerticaSparkTalend
Telecom Pakistan

DDSL Data Lake & Fraud Management System

Jazz (70M+ Customers)

Deployed HDP, HDF, and Cassandra clusters. Designed data lake on NiFi, exposed data via REST API with Cassandra as the performance layer. Also designed Fraud Management System architecture on 15-node HDP cluster with Pentaho BI suite.

HDP / HDFApache NiFiCassandraSpark ScalaKerberosPentaho
Telecom Pakistan

DWH Migration to Big Data

PTCL (Pakistan's Largest ICT Company)

Led full DWH-to-Hadoop migration as Technical Lead. Deployed 6-node CDH on Oracle Big Data Appliance, implemented Oracle Golden Gate CDC, built NiFi ETL flows Source→BDA→Exalytics, and enforced enterprise Kerberos + Sentry security.

Cloudera CDHOracle Golden GateApache NiFiSqoopKerberosSentry
Government UAE

GenAI / RAG Enterprise Chatbot

Roads & Transport Authority (RTA) Dubai

Designed and deployed a Generative AI chatbot powered by Retrieval-Augmented Generation (RAG) on Dataiku DSS. Integrated enterprise knowledge bases for internal Q&A, document retrieval, and decision support using LLM orchestration and vector search.

Dataiku DSSGenerative AIRAGLLMVector SearchPython
Government UAE

Parking Demand Forecasting Model

Roads & Transport Authority (RTA) Dubai

Built and deployed a production machine learning forecasting model on Dataiku DSS to predict parking demand across Dubai. Engineered features from IoT sensors, historical usage, and event data; managed full ML lifecycle from data preparation to model monitoring.

Dataiku DSSPythonML LifecyclePySparkCloudera CDPIoT Data
Government UAE

Statistical Data Platform

Statistics Centre — Abu Dhabi (SCAD)

Designed and developed end-to-end ETL pipelines using Azure Data Factory and Databricks (PySpark/SQL) to process data at scale. Implemented Delta Lake bronze-silver-gold architecture for ACID transactions, schema evolution, and optimized query performance.

DatabricksAzure ADFPySparkDelta LakePowerBIAzure Synapse
Defence Pakistan

National Security Project — Person 360

Islamabad Police

High-security defence project integrating data sources from across the country into a Hadoop platform for national-level analytics and intelligence. Deployed 14-node HDP cluster with Sqoop ingestion and Elasticsearch for distributed search.

HDP 2.6SqoopSpark ScalaElasticsearchHiveQL

Certifications

Dataiku Core Designer Certificate

Dataiku

2026

Certified Project Management Professional (PMP)

Project Management Institute

May 2025

Microsoft Certified: Azure AI Solution AI-102

Microsoft

Dec 2024

Red Hat Certified System Administrator (RHCSA)

Red Hat

Aug 2024

Microsoft Certified: Azure Data Engineer Associate DP-203

Microsoft

Feb 2024

Informatica Certified Developer 10.2

Informatica

Jan 2023

CDP Certified Administrator — Private Cloud Base

Cloudera

Sep 2022

Latest Articles

Let's Work Together

Have a data challenge? Let's discuss how I can help you build scalable, governed data solutions.

Location Dubai, United Arab Emirates