Data Engineer (Remote)
Truck Stop
Illinois
United States
Information Technology
(No Timezone Provided)
Truckstop.com is hiring a Data Engineer
Remote: Any Location
Position Summary:
As members of the Application Services team, Data engineers are responsible for creating and maintaining data pipelines, assembling complex datasets, and extracting, transforming, and loading data from a variety of data sources. Data engineers consult with development squads and data scientists to deliver product features and with cloud database administrators to resolve data related technical issues. Data engineers also identify and implement process improvements for greater reliability and performance.
Data engineers work as members of a Center of Excellence and participate in projects to ensure that Truckstop has hygienic, accurate, data available at the point of need.
Data Engineers may troubleshoot production issues and respond to incidents.
Essential Job Functions:
* Create, automate, and maintain data pipelines. Assemble complex data sets that meet business requirements.
* Extract, transform, and load data from a variety of data sources using SQL and big data tools and technologies.
* Identify, design, and implement internal process improvement including automating manual processes, optimizing data delivery, re-designing infrastructure for greater reliability and performance.
* Write and optimize queries, write lambda functions, perform source to target mapping and data modeling.
* Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
* Work as a member of the data engineering chapter on a development squad to deliver product features and resolve data related technical issues.
* Work as a member of the data management center of excellence on data quality projects.
* Work with information security to keep our data secure.
* Work with data architects, database administrators, and data engineers to continually improve our data systems.
* Work with IT operations to resolve data related technical issues and respond to data related major incidents.
* Create data tools to assist data analysts and data scientists in building and optimizing our product into an innovative industry leader.
* Troubleshoot operations and database issues for internal and external customers.
* Perform incident resolution and root cause analysis of critical outages. Implement solutions to systematic failures. Provide on-call support, including after hours.
* Assist with documentation of the environments that support our products.
* Other duties as assigned.
Position Requirements:
* Bachelor's degree or equivalent professional experience required; Computer Science or Engineering preferred.
* Strong experience in SQL and Python.
* Experience with relational SQL and NoSQL Databases; MS SQL, PostgreSQL, Snowflake, MongoDB, MySQL, MariaDB, Elastic stack.
* Experience building and optimizing reliable, idempotent data pipelines (streaming or batch) using ETL tools such as Matillion, Apache NiFi, Kafka and Spark.
* A successful history applying software engineering principles to build data infrastructure tools/libraries to automate and scale data pipelines.
* Understanding of applying logging and metrics to monitor and detect data pipeline performance issues.
* Solid understanding of Data warehousing theory and concepts.
* Working knowledge of data modeling and source to target mapping with a demonstrated understanding of data schema design and indexes, when to apply fact-based vs. time-based vs. relational and when to apply NoSQL approaches such as key-value, document, and column data stores.
* Understanding of Statistical and Machine Learning (ML) terminology with experience deploying and monitoring ML-models in a production environment.
* Experience working in a modern cloud-based environment a plus; Azure, AWS, Docker, Kubernetes. Data Warehouse
* Experience with Linux.
* Familiarity with Kubernetes, Lambda functions, Cloud Watch, SNS, S3, Glacier.
Physical Demands:
Minimum physical exertion. While performing the duties of this position, the employee is frequently required to sit, communicate, reach and manipulate objects, tools or controls. The position requires mobility. Duties involve moving materials weighing up to 5 pounds on a regular basis. Manual dexterity and coordination are required over 75% of the work period while operating equipment such as computer keyboard and mouse.
Work Environment:
The working conditions will vary between an office environment and a remote home environment. The ideal protected indoor work environment is a clean, non-hazardous work area. The noise level in the work environment is typical of most office or remote home environments with telephones, personal interruptions, and background noises.
Truckstop.com provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.
Data Engineer (Remote)
Truck Stop
Illinois
United States
Information Technology
(No Timezone Provided)
Truckstop.com is hiring a Data Engineer
Remote: Any Location
Position Summary:
As members of the Application Services team, Data engineers are responsible for creating and maintaining data pipelines, assembling complex datasets, and extracting, transforming, and loading data from a variety of data sources. Data engineers consult with development squads and data scientists to deliver product features and with cloud database administrators to resolve data related technical issues. Data engineers also identify and implement process improvements for greater reliability and performance.
Data engineers work as members of a Center of Excellence and participate in projects to ensure that Truckstop has hygienic, accurate, data available at the point of need.
Data Engineers may troubleshoot production issues and respond to incidents.
Essential Job Functions:
* Create, automate, and maintain data pipelines. Assemble complex data sets that meet business requirements.
* Extract, transform, and load data from a variety of data sources using SQL and big data tools and technologies.
* Identify, design, and implement internal process improvement including automating manual processes, optimizing data delivery, re-designing infrastructure for greater reliability and performance.
* Write and optimize queries, write lambda functions, perform source to target mapping and data modeling.
* Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
* Work as a member of the data engineering chapter on a development squad to deliver product features and resolve data related technical issues.
* Work as a member of the data management center of excellence on data quality projects.
* Work with information security to keep our data secure.
* Work with data architects, database administrators, and data engineers to continually improve our data systems.
* Work with IT operations to resolve data related technical issues and respond to data related major incidents.
* Create data tools to assist data analysts and data scientists in building and optimizing our product into an innovative industry leader.
* Troubleshoot operations and database issues for internal and external customers.
* Perform incident resolution and root cause analysis of critical outages. Implement solutions to systematic failures. Provide on-call support, including after hours.
* Assist with documentation of the environments that support our products.
* Other duties as assigned.
Position Requirements:
* Bachelor's degree or equivalent professional experience required; Computer Science or Engineering preferred.
* Strong experience in SQL and Python.
* Experience with relational SQL and NoSQL Databases; MS SQL, PostgreSQL, Snowflake, MongoDB, MySQL, MariaDB, Elastic stack.
* Experience building and optimizing reliable, idempotent data pipelines (streaming or batch) using ETL tools such as Matillion, Apache NiFi, Kafka and Spark.
* A successful history applying software engineering principles to build data infrastructure tools/libraries to automate and scale data pipelines.
* Understanding of applying logging and metrics to monitor and detect data pipeline performance issues.
* Solid understanding of Data warehousing theory and concepts.
* Working knowledge of data modeling and source to target mapping with a demonstrated understanding of data schema design and indexes, when to apply fact-based vs. time-based vs. relational and when to apply NoSQL approaches such as key-value, document, and column data stores.
* Understanding of Statistical and Machine Learning (ML) terminology with experience deploying and monitoring ML-models in a production environment.
* Experience working in a modern cloud-based environment a plus; Azure, AWS, Docker, Kubernetes. Data Warehouse
* Experience with Linux.
* Familiarity with Kubernetes, Lambda functions, Cloud Watch, SNS, S3, Glacier.
Physical Demands:
Minimum physical exertion. While performing the duties of this position, the employee is frequently required to sit, communicate, reach and manipulate objects, tools or controls. The position requires mobility. Duties involve moving materials weighing up to 5 pounds on a regular basis. Manual dexterity and coordination are required over 75% of the work period while operating equipment such as computer keyboard and mouse.
Work Environment:
The working conditions will vary between an office environment and a remote home environment. The ideal protected indoor work environment is a clean, non-hazardous work area. The noise level in the work environment is typical of most office or remote home environments with telephones, personal interruptions, and background noises.
Truckstop.com provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.