Wayfair Careers

Competitive Intelligence Engineer

Boston, MA

Job Description

About Wayfair Inc.

Wayfair believes everyone should live in a home they love. Through technology and innovation, Wayfair makes it possible for shoppers to quickly and easily find exactly what they want from a selection of more than 10 million items across home furnishings, décor, home improvement, housewares and more. Committed to delighting its customers every step of the way, Wayfair is reinventing the way people shop for their homes – from product discovery to final delivery.

The Wayfair family of sites includes:

  • Wayfair, an online destination for all things home
  • Joss & Main, where beautiful furniture and finds meet irresistible savings
  • AllModern, unbelievable prices on everything modern
  • Birch Lane, a collection of classic furnishings and timeless home décor
  • Perigold, unparalleled access to the finest home décor and furnishings

Wayfair generated $4.7 billion in net revenue for the twelve months ended December 31, 2017. Headquartered in Boston, Massachusetts with operations throughout North America and Europe, the company employs more than 7,700 people.

Wayfair's Pricing & Profitability Engineering team develops algorithms and tools to optimize and manage profit, revenue, and price perception. We are seeking an experienced, creative, and determined senior engineer to drive our Competitive Intelligence data acquisition efforts.  The mission of our Competitive Intelligence team is to define, gather, analyze and distribute intelligence about products and competitors to enable market-based assessments of Wayfair’s performance, guide targeted interventions, and track progress filling competitive gaps.  This is a unique opportunity to collaborate with our Competitive Intelligence business partners and design techniques to gather and store a vast amount of competitive information at scale.

Working within a quickly growing team at Wayfair, you will utilize your expertise in web design, JavaScript, and web scraping to ensure rapid and consistent intelligence gathering.  The intelligence you gather will form the foundation of a valuable data system that will drive major business decisions across every major department at the company.  The scope of impact is broad and ambitious; we want you to take pride and ownership in what you’re building - a truly cutting edge Competitive Intelligence system unmatched in the industry.


Recent Pricing & Profitability Projects:

  • A proprietary web scraper built on scrapy, an open source framework
  • Leveraging distributed technologies (Kafka, Spark, Hadoop) to scale data processing and storage
  • Creating in-house software analytics dashboards enabling swift business decision capabilities


  • Analyze competitor sites to understand their structure and flow
  • Acquire data from millions of webpages
  • Develop a strategy to avoid web-scraping detection and decrease blocking
  • Monitor real-time indicators to ensure we are quickly resolving blockages
  • Create and maintain best practices for data acquisition for various types of competitor sites
  • Design competitor data domain schema
  • Write test-driven Python code using Docker containers and GitLab CI
  • Work closely with the Data Science team to normalize, classify and continually improve upon our growing data system

Job Requirements:

  • 4 – 6 years of full stack web development experience and web scraping
  • Passion about deconstructing and analyzing webpages, with a focus on efficient and effective ways to extract information
  • Demonstrated track record of working and assessing quality on large amounts of data
  • Strong organizational skills and a fine attention to detail
  • Strongly collaborative and effective in a cross-functional environment


What we offer:

  • An opportunity to join a rapidly growing team, with strong senior executive sponsorship
  • Become a key leader at a unique time, developing Wayfair’s Competitive Intelligence system from the ground up
  • The ability to gain a detailed understanding across many areas of the business, and drive impact across every key strategic lever at Wayfair
  • A collaborative, forward-thinking, and entrepreneurial team that continuously pushes the envelope

About Us:

Wayfair is one of the world’s largest online destinations for the home. Whether you work in our global headquarters in Boston or Berlin, or in our warehouses or offices throughout the world, we’re reinventing the way people shop for their homes. Through our commitment to industry-leading technology and creative problem-solving, we are confident that Wayfair will be home to the most rewarding work of your career. If you’re looking for rapid growth, constant learning, and dynamic challenges, then you’ll find that amazing career opportunities are knocking.

No matter who you are, Wayfair is a place you can call home. We’re a community of innovators, risk-takers, and trailblazers who celebrate our differences, and know that our unique perspectives make us stronger, smarter, and well-positioned for success. We value and rely on the collective voices of our employees, customers, community, and suppliers to help guide us as we build a better Wayfair – and world – for all. Every voice, every perspective matters. That’s why we’re proud to be an equal opportunity employer. We do not discriminate on the basis of race, color, ethnicity, ancestry, religion, sex, national origin, sexual orientation, age, citizenship status, marital status, disability, gender identity, gender expression, veteran status, or genetic information.

Requisition ID: 23599