I’m newbie on big data and web development.
I’m curious to know what are the technologies used to develop the e-learning platform of Dataquest.
Based on some research findings,
dataquest.io runs on wordpress cms.
Listed technologies for the front end portion of full stack.
Back end stack is unknown.
Load balancing is unknown.
What runs on dataquest.io
PluginsThrive Leads, Ank Prism For, Posts Data Table, Q2w3 Fixed Widget, Thrive Visual Editor, Thrive Headline Optimizer, Autoptimize, Thrive Ultimatum, Search Filter, Svg Support, Latex2html, Onesignal Free Web Push Notifications
Google Analytics UA
Twitter Emoji (Twemoji)
Sales and Marketing
Google Tag Manager
Font FamilyIngra, Sans Serif
What runs on app.dataquest.io
Build CI Systems
Font FamilyMerriweather, Serif
Ok, interesting! But I want to know the back-end language on app.dataquest.io.
Hi @credo, thanks for the fun question! (And sorry I missed it for 6 months!) I work on the engineering team at Dataquest, so I can answer this with some authority.
Here’s the tech stack that our production application uses:
- React (frontend)
- Discourse (where we are right now! – the community)
- Wordpress (dataquest.io)
- Cloudflare (DNS, CDN, threat protection)
- NGINX (running in Kubernetes)
- Kubernetes (GKE in particular - this is where the application runs)
- Django, Django REST framework, GraphQL (backend)
- Celery with Redis as message broker (async application tasks)
- Postgres (database)
- Google cloud storage, AWS S3 (various static resource storage)
- Docker for our code running sandboxes
For infrastructure management, and CI/CD:
- Terraform (infrastructure management)
- Helm & Kubectl (application deploys)
- Jenkins (CI/CD server, automating various tasks)
We also use the following for local dev and content authoring environments:
- Docker again!
- Many/most developers use PyCharm
- Content authors use Jupyter notebooks
And for analytics:
- AWS Redshift
- Stitch data
- Mode analytics
- AWS Lambda with Python (for some simple ETL tasks)
Thanks Darla! So, the platform app.dataquest.io is developed in Python with django. What is your participation in these technologies’s using?
@credo We’re a very small team, so everyone actually contributes at least a bit across the entire stack, as necessary. However, my job title is “Infrastructure Engineer,” and I also have a background as a data engineer, so I contribute most often on infrastructure and CI/CD code / tooling, monitoring, and also some data engineering work to support analytics.
Also, Dataquest has a very “data first” culture (unsurprising?), so basically everyone across the company (me included!) does at least some analytics work now and then.
I personally try to stay away from frontend code as much as possible – but occasionally it is necessary, unfortunately.
Ok Darla, please give me a precision: Django are still used for developing app.dataquest? Why Django for this?
@credo yes, we use django for
I didn’t work at Dataquest yet when that was decided (it was just Vik did then!), but I would guess it was chosen just because it’s the most popular Python web framework, and Vik wanted to use Python.
This would be a very overloaded question, but if you don’t mind answering if possible -
Would you be able to give some tips/suggestions on how someone could create like a toy version of the website using the tech stack you mentioned? To be able to better their skills as a software developer, broadly speaking?
I would like to understand how all of those things connect together and get better at software development, so I am trying to think of projects which can help develop those skills along with what I am learning through DataQuest. These might not be strictly relevant to Data Science, but I would still like to learn more.
Do you think creating a toy/very tiny version of the website by yourself using that tech stack, over a period of time, is possible (without it being too expensive) for learning purposes? If yes, then how could one go about it?
@the_doctor - I think there are a lot of “moving parts” for Dataquest’s platform, and it might be better to pick a simpler project to start with – in particular, if you’re interested in learning web development and infrastructure tools, probably one without user code running and answer checking (unless those things interest you specifically of course!).
However, it’s also not unachievable, if you’re really committed to it!
I would suggest to start with a Django + React web application, and see how far you can get from there.
Here’s a simple tutorial to get started with Django & React (I just found that by googling “django react tutorial,” but it looks reasonable to me).
After that, the next step is probably to build Docker images for each component, and get everything running locally using
If you want to (eventually) test out on AWS or GCP, they both have reasonable free tier / trial offerings - it’s likely that if you’re careful, you can test it out for free (or nearly free).
With which tool has been made the code editor in the learning interface of DQ? I Just want to understand how your designer have made it very amazing!