Lead Software Engineer, Data Engineering
Company: S&P Global
Location: Cambridge
Posted on: March 20, 2023
|
|
Job Description:
The Role: Data Engineer
Location: Team is in Boston, but is available for remote or on-site
throughout CST and EST Time zones
GL (for internal use only): 11
Panjiva is a data-driven technology company that uses machine
learning to provide powerful search, analysis, and visualization of
billions of shipping records from nearly every country in the
world. More than 3,000 customers in over 100 countries, ranging
from Fortune 500 companies and startups to government agencies and
hedge funds, rely on our platform for supply chain intelligence. In
global trade, better insight means better decision making and
stronger connections between companies and governments across the
globe.
Recognizing Panjiva's cutting-edge technology, S&P Global
acquired Panjiva in 2018. This acquisition has grown our resources,
dramatically expanded our access to data, and accelerated our
growth plans.
People are Panjiva's greatest strength - join our engineering team
as we map out a key part of the world economy!
Job Description
As a data engineer on our team, you will play a key role in
developing our next-generation data science infrastructure and
underlying core technologies. You will work with Panjiva's
world-class data scientists, analysts, and engineers to create
products that solve important real-world business problems in a
collaborative, fast-paced, and fun environment.
You'll work closely with our data science team to develop new
platforms, infrastructure, and tools that will allow for machine
learning applications at production scale over ever-growing
datasets.
You'll design and leverage distributed computing technologies, data
schemas, and APIs to construct data science pipelines. In addition,
you'll be expected to participate in augmenting our infrastructure
to seamlessly integrate new data sets through constant R&D of
the technologies and systems we use.
Join us in building the next generation of products as we continue
to deliver valuable and actionable insights to decision-makers in
the $15 trillion global trade industry.
Responsibilities
Architect and implement distributed systems that perform complex
transformations, processing, and analysis over very large scale
datasets
Develop processes to monitor and automate detection of quality
regressions in raw data or in the output of Panjiva's machine
learning models
Working with our data scientists to turn large-scale messy,
diverse, and often unstructured data into a source of meaningful
insights for our customers
Optimizing slow-running database queries and data pipelines
Helping enhance our search engine, capable of running sophisticated
user queries quickly and efficiently
Building internal tools and backend services to enable our data
scientists and product engineers to improve efficiency
Qualifications
B.S., M.S., or Ph.D. in Computer Science (or a related field) or
equivalent work experience
7+ years of experience working with data-at-scale in a production
environment
Experience designing and implementing large-scale, distributed
systems
Experience in multi-threaded software development (or some form of
parallelism)
Significant performance engineering experience (e.g., profiling
slow code, understanding complicated query plans, etc.)
Solid understanding of core algorithms and data structures,
including the ability to select (and apply) the optimal ones to
computationally expensive operations over data-at-scale
Strong understanding of relational databases and proficiency with
SQL
Deep knowledge of at least one scripting language (e.g., Python,
Ruby, JavaScript)
Deep knowledge of at least one compiled language (e.g., Scala, C++,
Java, Go)
Experience developing software on Linux-based operating systems
Experience with distributed version control systems
Nice-to-Haves
Familiarity with relational database internals (especially
PostgreSQL)
Proficiency with cloud computing platforms, specifically AWS
Working knowledge of probability & statistics
Contributions to open-source software
Experience building customer-centric products
Compensation/Benefits Information (US Applicants Only):
S&P Global states that the anticipated base salary range for
this position is $85,300 - $170,000. Base salary ranges may vary by
geographic location.
In addition to base compensation, this role is eligible for an
annual incentive bonus plan.
This role is eligible to receive additional S&P Global
benefits. For more information on the benefits we provide to our
employees, visit https://www.spgbenefitessentials.com/newhires
.
About S&P Global
S&P Global delivers essential intelligence that powers decision
making. We provide the world's leading organizations with the right
data, connected technologies and expertise they need to move ahead.
As part of our team, you'll help solve complex challenges that
equip businesses, governments and individuals with the knowledge to
adapt to a changing economic landscape.
S&P Global Market Intelligence partners with customers to
broaden their perspective and operate with confidence by bringing
them leading data sources and technologies that embed insight in
their daily work.
We pride ourselves on our agility and diversity, and we welcome
requests to work flexibly. For most roles, flexible hours and/or an
element of remote working are usually possible. Please talk to us
at interview about the type of arrangement that is best for you. We
will always try to be adaptable wherever we can.
Equal Opportunity Employer
S&P Global is an equal opportunity employer and all qualified
candidates will receive consideration for employment without regard
to race/ethnicity, color, religion, sex, sexual orientation, gender
identity, national origin, age, disability, marital status,
military veteran status, unemployment status, or any other status
protected by law. Only electronic job submissions will be
considered for employment.
If you need an accommodation during the application process due to
a disability, please send an email to:---
EEO.Compliance@spglobal.com ---and your request will be forwarded
to the appropriate person.---
US Candidates Only: The EEO is the Law Poster
http://www.dol.gov/ofccp/regs/compliance/posters/pdf/eeopost.pdf---
describes discrimination protections under federal law.
20 - Professional (EEO-2 Job Categories-United States of America),
IFTECH202.2 - Middle Professional Tier II (EEO Job Group), SWP
Priority - Ratings - (Strategic Workforce Planning)
Job ID: 277527
Posted On: 2023-03-09
Location: Virtual, California, United States
Keywords: S&P Global, Cambridge , Lead Software Engineer, Data Engineering, IT / Software / Systems , Cambridge, Massachusetts
Click
here to apply!
|