Softway is a business-to-employee solutions company.We work with businesses to impact the everyday behaviors, mindsets, and attitudes and help drive valuable changes within.Faster innovation and more inclusion to lead cultural and digital transformations and make human-focused technology to propel businesses. Our solutions include culture & inclusion services and products, technology experiences, and communications. Ultimately, we believe love is a business strategy.
Learn more:
Visit us: https://www.softway.com/
Our Podcast : https://www.softway.com/laabs
Buy our book: https://www.loveasabusinessstrategy.com/
Who are we looking for ?
Softway is on the lookout for a Data Engineer in Python who is passionate about building products that our customers love. You will join a dynamic and fast-paced environment and work with cross-functional teams to design, build and roll-out products that deliver the company’s vision and strategy.
We're seeking a Python-savvy Data Engineer with a growing interest in big data technologies to join our dynamic team. You'll use Python to build essential data pipelines, wrangle datasets, and collaborate with our data science team. This is a fantastic opportunity to gain experience with PySpark and other in-demand tools while contributing to our data-driven initiatives.
Responsibilities
- Design and develop Python-based ETL/ELT processes, incorporating PySpark for larger datasets.
- Clean, transform, and prepare data for analysis using Python (Pandas, NumPy) and PySpark.
- Work with structured, semi-structured, and unstructured data from various sources
- Write SQL queries to effectively interact with relational databases.
- Collaborate effectively with team members from different disciplines, such as data science and business intelligence, to design and implement data solutions based on provided specifications.
- Explore and utilize cloud services for data storage and processing on AWS
- Learn and contribute to best practices in data pipeline development and maintenance considering optimizing performance and ensuring scalability.
- Lead development of entire data systems/subsystems and implement data pipelines
- Contribute to data system design and architecture along with working closely with various engineers to ensure that designs align with overall data architecture and scalability requirements
- Engage in client requirement gathering, solution presentation and feedback gathering to revise the solutions.
- Mentor team members and share knowledge fostering a culture of mentorship within the team, where knowledge and expertise are freely exchanged.
Requirements
- Basic understanding of PySpark for distributed data processing.
- Proficiency in SQL for querying and working with relational data.
- Familiarity with data warehousing and ETL/ELT principles.
- Willingness to learn and grow within a collaborative environment.
- Collaborate effectively with cross-functional teams to translate business requirements into solutions.
- Communicate technical concepts to diverse audiences
- Stay updated on industry trends and best practices
- Propose innovative solutions for business/technical challenges
- Demonstrate expertise and contribute to technical discussions
- Set up coding standards, best practices, and the overall architecture of the system for data solutions
- Drive innovation within the data engineering team and the organization. Identify areas for improvement and propose innovative solutions to address business needs or technical challenges.
- Should be leading by example in experimenting with new technologies and methodologies, evaluating their feasibility and potential impact
Great to Have
Passion to stay up-to-date with technology and trends
Expertise in PySpark.
Knowledge on different cloud services for data storage, processing and analytics from multiple cloud providers.
Why should you work at Softway?
Softway’s cross-functional team structure, varied domains and flat hierarchy offer much exposure and learning from a bird's eye-view. You’d love the interaction with our talented folks. Softway gives you ample learning opportunities, pushing you while having your back. We work hard and party harder !! We encourage you to voice your opinions and yes, we actually listen. We take pride in having an ego-less workforce, it allows us to focus on what we’re good at - bringing solutions to life. In addition to a competitive salary and great work culture, we offer GREAT FLEXIBILITY with Flexible Time Offs.