Selahattin Gungormus
Verified Expert in Engineering
Data Engineer and Developer
Selahattin是一名数据工程师,拥有多年使用开源技术构建可扩展数据集成解决方案的实践经验. 他擅长使用Hadoop等分布式处理平台开发数据应用程序, Spark, and Kafka. Selahattin在AWS和Azure等云架构类型方面也有实践经验, as well as developing microservices using Python and JavaScript frameworks
Portfolio
Experience
Availability
Preferred Environment
Apache Airflow, Visual Studio Code (VS Code), Apache Spark, Amazon Web Services (AWS), Azure, Jupyter Notebook
The most amazing...
...我所做的是构建一个利用Apache Spark进行数据处理的产品,并且可以通过拖放可视化界面进行操作.
Work Experience
Lead Data and Back-end Engineer
Afiniti
- Built a highly scalable, containerized data integration platform using Spark, Docker/Kubernetes, Python, and Greenplum database.
- Wrapped up whole data pipeline procedures in an easy-to-deploy templating system, capable of running at scale with good performance. That effort made the data pipeline process 70% faster.
- Created data models and pipelines for the application, resulting in powering dashboard reports with over 10 million events.
- Established and standardized CI/CD pipeline processes across the team using Jenkins, Bitbucket, and Kubernetes.
- Built and maintained an app's back-end service using Node.js, JavaScript, and GraphQL.
Senior Data Engineer
Iyzico
- 通过使用Airflow创建新的技术堆栈,重新设计和优化了现有的数据管道流程, Python, Spark, and Exasol database.
- 完成了超过300个数据流水线作业从Talend迁移到新数据平台,将每日ETL性能提高了60%(从8小时提高到3小时).
- 使用Spark Streaming和Kafka创建了从事务系统到仪表板的实时数据提要. 这个新功能提高了高峰时段性能监控的操作效率.
- 通过AWS进行整合,向AWS Redshift服务提供每日数据集市,为全球董事会提供每日报告.
Owner | Big Data Engineer | Instructor
Majestech
- 提供咨询和培训服务,以利用基于云的替代方案(如Amazon Web services和Azure)转换中小企业的数据架构.
- Delivered over ten data integration projects for businesses in the retail, banking, and telecommunications sectors. Transformed data integration processes to utilize cloud platforms such as AWS and Azure.
- 构建一个点击流数据应用程序,收集应用程序用户的网络痕迹,并以最小的延迟将它们存储在数据湖中. Used Kafka and Spark Streaming on AWS as the technology base.
- Launched a cloud-based data integration product: Integer8 on the AWS platform.
- 为想要利用Hadoop和Spark分布式处理能力的非开发人员数据专业人员构建了一个可视化界面.
- 与Cloudera合作提供大数据工程培训(超过20次培训).
- 使用Apache气流和S3连接器在AWS Snowflake Cloud DB上创建数据集成管道.
Data Engineer
i2i Systems
- 使用Python实现数据质量测试自动化,并使用Oracle元数据信息生成日常自动化任务,评估日常管道上可能存在的问题.
- 创建每日集成管道,为ODS和RDS层上的企业数据仓库提供数据.
- 为某电信运营商构建了一个市场优化项目的数据准备层. 来自3500多万用户的数据从五个不同的源系统收集到Oracle Data Integrator的非规范化数据结构中.
Experience
Integer8 Data Integrator
http://www.f6s.com/integer82015年,我和两名开发人员创建了自己的初创公司,在本地和国际市场上推出Integer8产品. I designed and led the development effort to make the product feasible for local SMEs. At the end of the first year, we deployed our platform to two different retail companies.
我成为了微软Azure在土耳其的云合作伙伴,并花了一年多的时间使Integer8符合Azure Marketplace的资格. 在这一努力的最后,Integer8成功成为Azure Marketplace的官方产品.
Data Warehouse Transformation for a Mobile Payment Company
作为负责新数据平台的数据工程师,设计并实现了整个数据管道流程. 我建立了一个CDC机制,从MySQL数据库到Kafka,提供一个pub/sub-event系统,用于近乎实时的集成. 然后,我准备了实时Spark Streaming作业来使用Kafka主题来刷新目标数据存储. 这有助于营销和运营团队监控系统上的工作量并发现异常情况.
所有数据源都被合并到Tableau报表层的两个主要数据集市中. 每日预聚合表帮助实时报表的执行速度比以前的实现快400%. 这也增加了整个组织的高级用户使用报告工具的动机.
Cloud ETL Automation on AWS
As the target database I used Amazon Redshift. 因此,单个事件从Amazon EventBridge发送到Lambda函数中,并累积到Redshift数据库中以供进一步分析.
Education
Bachelor's Degree in Computer Engineering
Istanbul Technical University - Istanbul, Turkey
Certifications
Cloudera Certified Developer for Apache Hadoop
Cloudera
Skills
Libraries/APIs
Spark Streaming, Node.js, Pandas
Tools
Apache Airflow, Amazon CloudWatch
Frameworks
Apache Spark, Hadoop, Spark
Paradigms
ETL, MapReduce, Database Design
Languages
Python, SQL, JavaScript, Scala, Snowflake, TypeScript
Storage
PL/SQL、数据库、数据管道、Redis、Greenplum、HDFS、HBase、Apache Hive、Amazon S3 (AWS S3)
Platforms
Azure, Apache Kafka, Oracle, Amazon Web Services (AWS), Docker, Visual Studio Code (VS Code), Kubernetes, Jupyter Notebook, Oracle Data Integrator 11g, Google Cloud Platform (GCP), AWS Lambda, Amazon EC2
Other
Data Modeling, Data Warehousing, Data Warehouse Design, ETL Development, Data Engineering, Data Architecture, Big Data Architecture, OOP Designs, Data Structures, Algorithms
How to Work with Toptal
在数小时内,而不是数周或数月,我们的网络将为您直接匹配全球行业专家.
Share your needs
Choose your talent
Start your risk-free talent trial
Top talent is in high demand.
Start hiring