Role: Senior Developer Hadoop
Location: Charlotte NC/ Pennington NJ
Fulltime Permanent
Job Description
Must Have Technical/Functional Skills
Primary Skill: Data Engineering, Platform Engineering or architecture roles. Deep Expertise in Pyspark.
Experience: 10+ yrs
Roles & Responsibilities
Bachelor's or master's degree in computer science or related field.
Required Hard and Soft Skills / Experience
Deep Expertise in PySpark, including performance tuning and optimization
Strong python development experience in large-scale distributed environment
Solid knowledge of Hadoop ecosystem (HDFS,Hive/Impala, YARN)
Proven experience designing and governing enterprise, regulatory facing data platforms.
Expertise in designing data lakes, ELT/ETL pipelines, batch and real time data processing solution
Proficiency in programming languages such as Java, Scala and SQL
Strong understanding of non-functional requirements and production support models
Clear written and verbal communication skills with ability to influence across organizations
Preferred Skills / Experience
Financial services experience, particularly in Market Surveillance, AML, Fraud, Or Risk Technology
Experience supporting regulatory or audit facing platforms
Kafka and Spark Structured streaming exposure
Familiarity with Orchestration tools(Airflow,Control-M,Oozie)
Knowledge of data governance, lineage, and data quality controls
Daily Responsibilities/Priorities
Enterprise & Surveillance Architecture
Define target state architecture and strategic roadmap for surveillance data processing platforms leveraging Hadoop,Spark, Pyspark and Python
Establish standard architectural patterns for alert generation, enrichment, aggregation and reporting pipelines.
Ensure architecture aligns with enterprise technology standards and Surveillance control expectation
Solution Design and Delivery Support
Translate surveillance business requirements(e.g, market misconduct detection, regulatory coverage) into scalable technical designs
Review and approve detailed technical designs, ensuring alignment with functional intent, regulatory requirements, and architectural standards.
Provide hands-on architectural guidance to engineering teams during development, testing and implementation
Design and standardization of Spark/Pyspark frameworks supporting surveillance alert generation and enrichment
Modernization and Optimization of large-scale Hadoop surveillance workloads to improve performance, stability and control coverage
Implementation of enterprise-consistent architecture patterns enabling audit readiness, lineage, and regulatory traceability
Support end-to-end delivery across the SDLC, minimizing rework and technical debt
Architecture Governance & Leadership
Participate in and lead architecture review forums and design walkthroughs
Mentor senior engineers and promote adoption of Standard framework and best practices
Influence enterprise surveillance platform strategy through architecture governance
Documentation & Communication
Maintain high-quality documentation, including business requirement, data definitions and process flows
Proactively communicate risks, dependencies and potential timeline impact to stakeholders
Non-Functional requirement and product Readiness
Fully Remote (UK based) | Competitive Salary + Bonus + Generous Equity | B Corp Certified TrustedHousesitters is a fast growing, direct-to-consumer marketplace that is redefining pet care and travel. Here at TrustedHousesitters, were on a mission to make travel...
...you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a IT Desktop Support Technician II to join our team in Arlington, Virginia (US-VA), United States (US). Job Summary: The Desktop Support Technician...
...company, is seeking a high-impact Key Account Manager to lead the Walmart Dolls IP business and drive profitable growth across one of... ...are sold in collaboration with the worlds leading retail and ecommerce companies. Since its founding in 1945, Mattel is proud to be a...
...for preparing, storing, rotating, and stocking of product. May assist in training of new Team Members, utilizing all learning... ...assigned by store, regional, or national leadership. Maximizes sales potential through effective and proper procedures for prepping,...
Job Description: HOURS: 7am-7 p.m.PAY: Negotiable DUTIES: Get children (two boys, 6 and 7) ready for school and handle drop off and pick ups. Prepare meals, clean up after the children, laundry, run errands, grocery shopping, etc. Help children with homework, bring...