The technology using by Dataquest

hi,
I’m newbie on big data and web development.
I’m curious to know what are the technologies used to develop the e-learning platform of Dataquest.

Based on some research findings,

dataquest.io runs on wordpress cms.

app.dataquest.io front end runs on react javascript framework.

Listed technologies for the front end portion of full stack.

Back end stack is unknown.

Load balancing is unknown.

What runs on dataquest.io

CMS

WordPress 5.2.2WordPress 5.2.2
ThemeIgnition
PluginsThrive Leads, Ank Prism For, Posts Data Table, Q2w3 Fixed Widget, Thrive Visual Editor, Thrive Headline Optimizer, Autoptimize, Thrive Ultimatum, Search Filter, Svg Support, Latex2html, Onesignal Free Web Push Notifications

Analytics

Google Analytics UA
Infer
Profitwell

Javascript Graphics

Twitter Emoji (Twemoji)

CDN

CloudFlare

Advertising

LinkedIn Insight
Facebook Pixel

Dev Tools

HTML5 Shiv

Javascript Frameworks

DataTables
jQuery 1.11.0
Lazysizes JS
embed JS
Masonry

Widgets

Facebook

Miscellaneous

Segment
Hellobar
Onesignal

Programming Language

PHP 7.2.19

Sales and Marketing

Yoast SEO

Tag Managers

Google Tag Manager

Communication

Intercom

Font

FontIngra
Font FamilyIngra, Sans Serif

What runs on app.dataquest.io

Captcha

reCAPTCHA

Miscellaneous

SegmentSegment
PingdomPingdom

Build CI Systems

webpackwebpack

Javascript Frameworks

Lo-dashLo-dash
React Reat

Font Script

TypekitTypekit

CDN

CloudFlare CloudFlare
Unpkg Unpkg

CDN JSCDN JS

Communication

Intercom Intercom

Font

FontMerriweather
Font FamilyMerriweather, Serif

2 Likes

Ok, interesting! But I want to know the back-end language on app.dataquest.io.

Hi @credo, thanks for the fun question! (And sorry I missed it for 6 months!) I work on the engineering team at Dataquest, so I can answer this with some authority. :slightly_smiling_face:

Here’s the tech stack that our production application uses:

  • React (frontend)
  • Discourse (where we are right now! – the community)
  • Wordpress (dataquest.io)
  • Cloudflare (DNS, CDN, threat protection)
  • NGINX (running in Kubernetes)
  • Kubernetes (GKE in particular - this is where the application runs)
  • Django, Django REST framework, GraphQL (backend)
  • Celery with Redis as message broker (async application tasks)
  • Postgres (database)
  • Google cloud storage, AWS S3 (various static resource storage)
  • Docker for our code running sandboxes

For infrastructure management, and CI/CD:

  • Terraform (infrastructure management)
  • Helm & Kubectl (application deploys)
  • Jenkins (CI/CD server, automating various tasks)

We also use the following for local dev and content authoring environments:

  • Docker again!
  • Many/most developers use PyCharm
  • Content authors use Jupyter notebooks
  • Github

And for analytics:

  • AWS Redshift
  • Segment.io
  • Stitch data
  • Mode analytics
  • AWS Lambda with Python (for some simple ETL tasks)
7 Likes

Thanks Darla! So, the platform app.dataquest.io is developed in Python with django. What is your participation in these technologies’s using?

@credo We’re a very small team, so everyone actually contributes at least a bit across the entire stack, as necessary. However, my job title is “Infrastructure Engineer,” and I also have a background as a data engineer, so I contribute most often on infrastructure and CI/CD code / tooling, monitoring, and also some data engineering work to support analytics.

Also, Dataquest has a very “data first” culture (unsurprising?), so basically everyone across the company (me included!) does at least some analytics work now and then. :slightly_smiling_face:

I personally try to stay away from frontend code as much as possible – but occasionally it is necessary, unfortunately. :joy:

4 Likes

Ok Darla, please give me a precision: Django are still used for developing app.dataquest? Why Django for this?

@credo yes, we use django for app.dataquest.io.

I didn’t work at Dataquest yet when that was decided (it was just Vik did then!), but I would guess it was chosen just because it’s the most popular Python web framework, and Vik wanted to use Python. :slightly_smiling_face:

2 Likes

Hey @darla!

This would be a very overloaded question, but if you don’t mind answering if possible -

Would you be able to give some tips/suggestions on how someone could create like a toy version of the website using the tech stack you mentioned? To be able to better their skills as a software developer, broadly speaking?

I would like to understand how all of those things connect together and get better at software development, so I am trying to think of projects which can help develop those skills along with what I am learning through DataQuest. These might not be strictly relevant to Data Science, but I would still like to learn more.

Do you think creating a toy/very tiny version of the website by yourself using that tech stack, over a period of time, is possible (without it being too expensive) for learning purposes? If yes, then how could one go about it?

1 Like

@the_doctor - I think there are a lot of “moving parts” for Dataquest’s platform, and it might be better to pick a simpler project to start with – in particular, if you’re interested in learning web development and infrastructure tools, probably one without user code running and answer checking (unless those things interest you specifically of course!).

However, it’s also not unachievable, if you’re really committed to it!

I would suggest to start with a Django + React web application, and see how far you can get from there.

Here’s a simple tutorial to get started with Django & React (I just found that by googling “django react tutorial,” but it looks reasonable to me).

After that, the next step is probably to build Docker images for each component, and get everything running locally using docker-compose.

If you want to (eventually) test out on AWS or GCP, they both have reasonable free tier / trial offerings - it’s likely that if you’re careful, you can test it out for free (or nearly free). :slightly_smiling_face:

3 Likes

Hello Darla,
With which tool has been made the code editor in the learning interface of DQ? I Just want to understand how your designer have made it very amazing! :wink: