At Quantifind we are looking for engineers with a passion for building highly performant, scalable systems. As Data Crawling and Acquisition Engineer (a.k.a., our Data Wrangler) you will be responsible for pulling in massive amounts of data from the web and a large number of structured and unstructured sources, and feeding it all into our big data machine.
We believe that the web is the greatest data source in the world, especially when coupled with exclusive high value data feeds that we have acquired. We are looking for like-minded souls to help us acquire, parse, and process this data to deliver tangible value to both our internal team and our customers. We know that is not always an easy task to manage, and requires creativity, tenacity, and perseverance; should you be equal to the challenge then we’d love to hear from you.
If you are looking to work at a company where you design, build, and feed the foundation of an entire data ecosystem, then Quantifind is for you.
Representative Skills and Tools
- Scala/Java (required).
- Storm, or other stream processing.
- Cassandra/MongoDB, SQL.
- IP proxy management.
- Standard web/API protocols (HTTP/REST, FTP, SOAP)
- Parsing of structured (JSON, XML) and unstructured (HTML, text) data.
- Efficient use of standard Social APIs (Open Graph, Twitter, Facebook, Tumblr,…)
- Familiarity with open source crawler technologies, job schedulers & thread management.
- PhantomJS, ZombieJS.
Quantifind is looking for a talented backend engineer to be the designer, owner, and maintainer of an ever growing collection of data. We need someone with an interest in schema design and storage mechanisms, and the drive to adapt to new sources quickly and intelligently.
As lead of our data team, you will present a unified API to our master configuration datastore that powers numerous cogs of our big data machine, and supports the development of new products and services which rely on this data. This is a critical role for Quantifind and requires a dedicated engineer to be the driving force behind innovation in this space. Whether you are an old pro looking to build on your experience, or a talented newcomer looking to implement your ideas - we’d love to hear from you.
Representative Skills and Tools
- Scala/Java/C++ or equivalent (required)
- Maintain existing Cassandra and MongoDB data schema.
- SQL and other standard database.
- Add new domain classes and maintain existing ones.
- Maintaining and adding new data serialization formats (JSON, Protobufs,…)
- Implement a data repository to list all available data incl. descriptions about types, freshness and field-level processing instructions.
- Schema/ontology design methods.
- Expose the data to other consumers in the system through APIs (e.g. REST)
- Creation of automated and manual data checks.
- Implementation of data cleaning & refining tools.
We’re on the lookout for front end engineers to help us build products that are on the cutting edge of usability, interaction and design. Ideal candidates should be comfortable taking ownership of their work and effectively searching for innovative solutions to the complex problems we encounter. You will be working in a high-communication environment where collaboration with the Design, Data Science and Back-End Engineering teams is effortlessly realized on a day-to-day basis in order to build world class software products. Having leaders in Data Science, Software Engineering, and Design are the three key pillars to our success. At Quantifind, big ideas trump titles. Experimentation with new technologies is in our lifeblood.
Desired skills include:
- Cool frameworks like Angular.js, Backbone.js, Underscore.js, etc.
- Data visualization experience using D3.js, SVG, Canvas, etc.
- Web app experience with Grails, Rails, Node.js, PhantomJS or a similar platform.
- Version control experience in Git or a similar system.
- Experience with Scrum, Agile Development, Pair Programming is a plus.
- Love of design or knowledge of design tools is a plus.
- Don’t see a technology you love? Tell us why we should know about it.
- Don’t have experience in something listed? We don’t mind if you’re a quick study.