Historically, there was no direct monetization involved in contributing to TDS (unless authors chose to join the partner program at our former hosting platform). As we establish TDS as an independent, self-sustaining publication, we’ve decided to change course, as it was important for us to reward the articles that help us reach our business goals in proportion to their impact.

How it works

The TDS Author Payment Program is structured around a 30-day window. Articles are eligible for payment based on the number of readers who engage with them in the first 30 days after publication.

Authors are paid based on three earning tiers:

• 25,000+ Views: The article will earn $0.1 per view within 30 days of publication: a minimum of $2,500, and up to $7,500, which is the cap for earnings per article.
• 10,000-24,999 Views: The article will earn $0.05 per view within 30 days of publication: a minimum of $500, and up to $1,249.
• 5,000-9,999 Views: The article will earn $0.025 per view within 30 days of publication: a minimum of $125, and up to $249.

A few important points to keep in mind: 

• Views are counted only if a reader stays on the page for at least 30 seconds, ensuring that the payouts reflect real engagement, not clicks.
• Articles with fewer than 5,000 views in 30 days will not qualify for payment.
• During these 30 days, articles must remain exclusive to Towards Data Science. After that, authors are free to republish or remove their articles.

Who can participate?

This program is available to every current TDS contributor, and to any new author who becomes eligible once an article reaches the first earning tier.

Participation in the program is subject to approval to ensure authentic traffic. We reserve the right to pause or decline participation if we detect unusual spikes or fraudulent activity. Additionally, payments are only available to authors who live in countries supported by Stripe.

Authors can submit up to four articles per month for paid participation.

Why we’re doing this

We built this program to create a transparent and sustainable system that pays contributors for the time and effort required to write great articles that attract a wide audience of data science and machine learning professionals. By tracking genuine engagement, we ensure that the best work gets recognized and rewarded while keeping the system simple and transparent.

We’re excited to offer this opportunity and look forward to supporting our contributors who keep Towards Data Science the leading destination in the data science community. 

How to contribute work?

We’re working swiftly to roll out an author portal that will streamline article pitches and feedback.

In the meantime, please send your upcoming article directly to our team using this form.

If you’re having an issue with our online form, please let us know via email (publication@towardsdatascience.com) so we can help you complete the process. Please do not email us an article that you have already sent via our form.

The post Announcing the Towards Data Science Author Payment Program appeared first on Towards Data Science.

• Submission Guidelines
• How To Submit Your Work
• How to get your article ready for publication!
• Adding and using images
• Longform posts, columns, and online books
• FAQ

Why become a contributor?

We are looking for writers to propose up-to-date content focused on data science, machine learning, artificial intelligence and programming. If you love to write about these topics, read on!

Reach a broader audience with your articles. We are one of the most popular data science sites in the world. TDS started as a publication on Medium, amassing more than 700k followers and becoming the most-read publication on the site. Now on a self-hosted platform, TDS is the leading destination in the data science community. 

Here are a few things we do to ensure your articles reach the largest audience possible:

• Our independent domain (towardsdatascience.com) provides better visibility and direct traffic to your work.
• We feature our best stories on our homepage, newsletter, and social media (LinkedIn, X, and more), and provide our authors with sophisticated publishing tools to better tell their stories. 
• We provide editorial support to help refine and amplify high-quality submissions.

Earn money with the Towards Data Science Author Payment Program. When publishing in TDS, our authors can decide to apply to our Payment Program, which enables them to earn from their work. You can read more about our Author Payment Program here.

Submission Guidelines

Before submitting your article, there are a few essential things you need to know. Make sure you read each point well, and that you understand them, as by submitting an article to TDS, you are agreeing to comply with all of them.

Please take a few minutes to familiarize yourself with our Author Terms and Conditions of Use — they govern the relationship between contributors and TDS.

Any article you share with us must be entirely your own original work; you can’t take other writers’ words and present them as your own, and we also don’t allow AI-generated text, even when you’re the one who prompted its creation.

How To Submit Your Work

Recently, TDS made a big leap toward independence by moving off Medium and launching our self-hosted platform. We’re working swiftly to roll out an author portal that will streamline article pitches and feedback.

In the meantime, please send your upcoming article directly to our team using this form.

If you’re having an issue with our online form, please let us know via email (publication@towardsdatascience.com) so we can help you complete the process. Please do not email us an article that you have already sent via our form.

Guidelines

How to get your article ready for publication!

We aim to strike a balance between innovating, informing and philosophizing. We want to hear from you! If you are not a professional writer, consider the following points when preparing your article. We want to publish high quality, professional articles that people want to read.

1. Is your story a story that needs to be told?

Before you start writing, ask yourself: is this story a story that needs to be told?

If you have read many articles addressing the same issue or explaining the same concept, think twice before writing another one. If you have a radical, new take on an old chestnut, we want to hear from you… but, we need you to persuade us that your article is something special that distinguishes itself from the pack and speaks to our audience.

Conversely, if your article addresses an underserved area or presents a new idea or method, that’s just what we are after!

2. What is your message?

Let us know what your main message is, right from the start. Give your piece a snappy introduction that tells us:

• What is your novel idea?
• Why should we care?
• How are you going to prove your point?

Once you’ve got that out of the way, you can be as conversational as you like, but keep calling back to the central message and give us a solid conclusion.

Remember though, Towards Data Science is not your personal blog, keep it sharp and on-topic!

3. On the internet, nobody knows you are a dog

You’ve got a new idea or a new way of doing things, you want to tell the community and start a discussion. Fantastic, that’s what we want too, but we’re not going to take for granted that you know what you are talking about or that we should uncritically believe what you say… you’ve got to persuade us (your audience) that:

• The subject matter is important
• There is a gap that needs to be filled
• You have the answer
• Your solution works
• Your idea is based on a logical progression of ideas and evidence
• If you are giving us a tutorial, tell us why people would need to use this tool and why your way is better than the methods already published.

You can do this by explaining the background, showing examples, providing an experiment or just laying out how data you have extracted from various sources allowed you to synthesise this new idea.

Are there arguments that counter your opinion or your findings? Explain why that interpretation conflicts with your idea and why your idea comes out on top.

4. Do you have a short title with an insightful subtitle?

If you scroll up to the top of this page, you will see an example of a title and subtitle. Your post needs to have a short title and a longer subtitle that tell readers what your article is about or why they should read it. Your header is useful for attracting potential readers and making your intentions clear. To remain consistent and give readers the best experience possible, we do not allow titles or subtitles written in all-caps. We also ask that you avoid profanity in both your title and subtitle.

When your subtitle is directly under the title and formatted correctly, it will show up in some post previews, which helps with your click-through rate. 

5. What makes your post valuable to readers?

A successful post has a clearly defined and well-scoped goal, and follows through on its promise. If your title tells us you’re going to unpack a complex algorithm, show the benefits of a new library, or walk us through your own data pipeline, make sure the rest of the post delivers.

Here are a few pointers to help you plan and execute a well-crafted post:

• 1. Decide what your topic is — and what it isn’t
  If you’re not sure what your post is going to be about, there’s very little chance your audience will when they read it. Define the problem or question your article will tackle, and stick to it: anything that doesn’t address the core of your post should stay out.

• 2. Create a clear plan
  With your topic in hand, sketch out a clear structure for your post, and keep in mind the overall structure it’ll follow. Remember that your main goal is to keep your reader engaged and well-oriented, so it’s never too early to think about formatting and how you’ll break down the topic into digestible sections. Consider adding section headings along the way to make your structure visible.

• 3. Use clear, action-driven language
  If you’re still finding your personal voice as a data-science author, a good place to start is keeping things clean, clear, and easy to follow.

If your article is full of neutral, generic verbs (like to be, have, go, become, make, etc.), try to mix in more precise action verbs. When it makes sense, use specific, lively descriptors instead of dull ones (for example, you could replace “easy” with “frictionless,” “accessible,” or “straightforward,” depending on the context).

There are few things editors appreciate more than a clean first draft, so don’t forget to proofread your post a couple of times before sharing it with TDS: look for spelling, punctuation, and grammar issues, and do your best to fix them. What we hope to offer to our readers are clear explanations, a smooth overall flow — pay attention to those transitions! — and a strong sense of what you’re aiming to achieve with your post.

If you’d like to expand your toolkit beyond the basics, the Internet is full of great writing resources. Here are a few ideas to help you get started:

• Writing for an academic journal: 10 tips. The Guardian
• 6 of the Best Pieces of Advice from Successful Writers
• Steven Pinker’s Sense of Style

• 4. Include your own images, graphs, and gifs
  One of the most effective ways to get your key points across to your readers is to illustrate them with your compelling visuals.

For example, if you’re talking about a data pipeline you built, text can only take you so far; adding a diagram or flowchart could make things even clearer. If you’re covering an algorithm or another abstract concept, make it more concrete with graphs, drawings, or gifs to complement your verbal descriptions. (If you’re using images someone else created, you’ll need to source and cite them carefully — read our image guidelines below for more details.)

A strong visual component will hook your readers’ attention and guide them along as they read your post. It will also help you develop a personal style as an author, grow your following, and draw more attention on social media.

6. Are your code and equations well displayed?

TDS readers love to tinker with the ideas and workflows you share with them, which means that including a code implementation and relevant equation(s) in your post is often a great idea.

To make code snippets more accessible and usable, avoid screenshots. Use WordPress’s code blocks & inline code

To share math equations with your readers, Embed.fun is a great option. Alternatively, you can use Unicode characters and upload an image of the resulting equation.

When you include code or an equation within your article, be sure to explain it and add some context around it so readers of all levels can follow along.

To learn more about using these embeds and others in your post, check out this resource.

7. Check your facts

Whenever you provide a fact, if it’s not self-evident, let us know where you learned it. Tell us who your sources are and where your data originated. If we want to have a conversation we all need to be on the same page. Maybe something you say will spark a discussion, but if we want to be sure we are not at cross purposes, we need to go back to the original and read for ourselves in case we are missing a vital piece of the puzzle that makes everything you say make sense.

8. Is your conclusion to the point and not promotional?

Please make sure that you include a conclusion at the end of your article. It’s a great way to help your readers review and remember the essential points or ideas you’ve covered. You can also use your conclusion to link an original post or a few relevant articles.

Adding an extra link to your author profile or to a social media account is fine, but please avoid call-to-action (CTA) buttons.

For your references, please respect this format:

[X] N. Name, Title (Year), Source

For example, your first reference should look like this:

[1] A. Pesah, A. Wehenkel and G. Louppe, Recurrent Machines for Likelihood-Free Inference (2018), NeurIPS 2018 Workshop on Meta-Learning

9. Are your tags precise enough?

The more specific your tags, the easier it is for readers to find your article and for us to classify and recommend your post to the relevant audience.

We may change one or two tags before publication. We would do this only to keep our different sections relevant to our readers. For instance, we would want to avoid tagging a post on linear regression as “Artificial Intelligence”.

10. Do you have an amazing image?

A great image attracts and excites readers. That’s why all the best newspapers always display incredible pictures.

This is what you can do to add a fantastic featured image to your post:

• Use Unsplash. Most of the content on Unsplash is fine to use without asking for permission. You can learn more about their license here.

• Take one yourself. Your phone is almost certainly good enough to capture a cool image of your surroundings. You might even already have an image on your phone that would make a great addition to your article.
• Make a great graph. If your post involves data analysis, spend some time making at least one graph truly unique. You can try R, Python, D3.js or Plotly.

If you decide to purchase a license for an image to be used in your article, please note that we only allow the use of images under a license that: (i) does not expire; and (ii) that can be used for commercial purposes on the TDS Publication. You are responsible for ensuring you comply with the license terms of use. You must also include a caption below the image, as follows, or as otherwise required by the license provider: “Image via [license provider’s name] under license to [your name].” Finally, please email us a copy of a receipt or other evidence of the purchased license, along with the corresponding license terms of use.

If you’ve chosen to create images for your article using an AI tool (like DALL·E 2, DALL·E, Midjourney, or Stable Diffusion, among others), it’s your responsibility to ensure that you’ve read, understood, and followed the tool’s terms. Any image you use on TDS must be licensed for commercial use, including AI-generated images. Not all AI tools permit images to be used for commercial purposes and some require payment to permit you to use the image.

The images you generate with AI tools cannot violate the copyright of other creators. If the AI generated image resembles or is identical to an existing copyrighted image or fictional character (like Harry Potter, Fred Flinstone etc.), you are not permitted to use it on TDS. Use your best judgment and avoid AI-generated images that copy or closely emulate another work. If in doubt, use an image search tool — like Google Lens, TinEye, or others — to check whether your images are too similar to an existing work. We may also ask that you provide details of the text prompts you used in the AI tool to confirm you did not use the names of copyrighted works.

Your text prompts cannot use the names of real people, nor can your images be used if they feature a real person (whether a celebrity, politician, or anyone else).

Please remember to cite the source of your images even if you aren’t legally obligated to do so. If you created an image yourself, you can add (Image by author) in the caption. Whichever way you decide to go, your image source should look like this:

Your image should both have the source and the link to that source. If you created an image yourself, you can add “Image by author”.

If you’ve created an image that was lightly inspired by an existing image, please add the caption “Image by Author, inspired by source[include the link].” If you’ve edited an existing image, please make sure you have the right to use and edit that image and include the caption “Image by source[include the link], edited with permission by the author.”

Danger zone: Do not use images (including logos and gifs) you found online without explicit permission from the owner. Adding the source to an image doesn’t grant you the right to use it.

11. Where did you get your data?

The Towards Data Science team is committed to the creation of a respectful community of data science authors, researchers, and readers. For our authors, this means respecting the work of others, taking care to honor copyrights associated with images, published material, and data. Please always ensure that you have the right to collect, analyze, and present the data you’re using in your article.

There are plenty of great sources of data that are freely available. Try searching university databases, government open data sites, and international institutions, such as the UCI Irvine Machine Learning Repository, U.S. Government, and World Bank Open Data. And don’t forget about sites that hold specific data relating to fields like physics, astrophysics, earth science, sports, and politics like CERN, NASA, and FiveThirtyEight.

TDS is a commercial publication. Before submitting your article to us, please verify your dataset is licensed for commercial use, or obtain written permission to use it. Please note that not all the datasets on the websites we’ve listed are fine to use. No matter where you obtain your data, we advise you to double-check that the dataset permits commercial use.

If you aren’t confident you have the right to use it for commercial purposes, consider contacting the owner. Many authors receive a quick, positive response to a well-constructed email. Explain how you intend to use the data, share your article or idea, and provide a link to TDS. When you receive permission, please forward a copy to us at publication@towardsdatascience.com.

This is especially important if you plan to use web scraping to create your own dataset. If the website does not explicitly allow data scraping for commercial purposes, we strongly recommend that you contact the website owner for permission. Without explicit permission, we won’t be able to publish your work, so please forward us a copy via email.

And sometimes, simple works best! If you just want a dataset to explain how an algorithm works, you can always create an artificial or simulated dataset. Here’s a quick tutorial, and an article that uses a simulated dataset you might find helpful.

Please remember to add a link to the site where the dataset is stored, and credit the owner/creator in your article. Ideally, this is done on first mention of the dataset, or in a resource list at the end of the article. Please carefully follow any instructions relating to attribution that you find on the site. If you have created your own artificial or simulated dataset, it is important to mention that too.

We know interpreting a license can be challenging. It is your responsibility to be certain that you can present your data and findings in an article published with TDS, but if you’re stuck, please reach out to our editorial team for assistance. We would rather work with you in the early stages of your project than to have to decline your completed article due to a dataset license issue.

13. Is your content original?

While we do accept content that has already been published (for example, on your personal blog or website), our focus is on promoting and sharing new and original content with our readers. That means that by publishing your article in TDS first (or exclusively), you have a greater chance to be featured on our publication, our social media channels, and in our newsletter.

We love original content because it’s something that our audience hasn’t seen before. We want to give as much exposure to new material as possible and keep TDS fresh and up-to-date.

Originality also means that you (and your coauthors, if any) are the sole creator of each and every element in your post. Any time you rely on someone else’s words, you have to cite and quote them properly, otherwise we consider it an instance of plagiarism. This applies to human authors, of course, but also to AI-generated text. We generally don’t allow any language created by tools like ChatGPT on TDS; if your article discusses these tools and you wish to include examples of text you generated, please keep them to a minimum, cite their source and the prompt you used, and make it very clear (for example, by using block quotes) where the AI-generated portions begin and end.

14. Did you get any feedback before submitting your post?

Get into the habit of always asking a friend for feedback before publishing your article. Having worked so hard on that article, you wouldn’t want to let a silly mistake push readers away.

15. Has your Author profile been completed correctly?

Please include your real name, a photo, and a bio. We don’t publish posts from anonymous writers — it’s easier to build trust with readers when they associate your words with an actual person.

Use your profile to introduce yourself, your expertise, your and achievements — optimizing it will help you develop a meaningful relationship with your audience beyond a single post. 

If you are a company and would like to publish with us, please note that we almost exclusively publish articles submitted directly from the author.

16. Are you getting better?

Take a minute to reflect on the work you have been doing so far, and the current article you wish to publish. What value are you bringing, and to whom? In which ways are this article better or worse than the ones you previously published?

Longform posts, columns, and online books

Have a lot to say? Good. We love to dive deep into complex topics, and so do our readers. Here’s how you can publish longform posts, columns, and online books on TDS.

Longform posts

We love long reads! If your article’s reading time is shorter than 25 minutes, we recommend that you don’t break it into multiple pieces — keep it as-is. A single post makes it easier for readers to search and find all the information they need, and less likely that they’ll miss an important part of your argument.

To create a smoother reading experience, you can add a table of contents to orient your audience around your post. Adding high-quality images and lots of white space is always a good idea, too — a long text doesn’t have to be a wall of text.

We regularly add the most engaging and thoughtful longform posts to our Deep Dives page.

Columns

If your post’s reading time exceeds 25 minutes, or if you plan to focus on the same topic over multiple articles and a longer stretch of time, you can create your own TDS column. All it takes are three steps:

1. Add a custom tag to your post. This tag needs to be unique and reflect the theme of your project. Every time you publish a post with that tag, it will be added to your column’s landing page: towardsdatascience.com/tagged/[your-tag].
2. Add a kicker to your post. It’s like adding a subtitle but above your title.
3. Link your kicker to your column’s landing page.

You can create a TDS column and invite multiple authors to contribute. Just let your colleague(s) know which tag you decided to use so that they can add the same one to their articles. Here are some examples from our team.

Online Books

A column is a great format to use if you have an open-ended topic that you plan to write about for a while. If, on the other hand, your idea has a finite, defined scope and a clear sense of progression from one post to the next, you may want to create a series of articles that feels more like an online book. Here is the format we recommend using.

Keep the reading time of each article — or “chapter” — between 12 to 25 minutes, and aim for a series that has at least 5 articles (but probably not more than, say, 16). You can add links to previous or subsequent items from within each article — for example, in the introduction and/or conclusion.

To publish your online book, you can submit all your articles to our editorial team in one go, or one by one as you finish working on each. We’ll review them and publish them as they come along. Let us know your post is part of a planned online book project.

Please ensure that each article or online book chapter follows the same guidelines and rules as any other post that TDS publishes. If you ever decide to sell or exclusively license your book to a third party publisher, you will have to make sure you have their consent to continue to publish the book with TDS. If you do not have such consent, it is your responsibility to remove your content from the TDS publication.

How do you submit an article?

To become a writer, please send your article using our form. Please note that a new author submission process is nearing launch, and this form will be retired upon it’s availability.

We aim to respond to authors as quickly as possible and to let them know whether or not we’ve accepted their articles. On rare occasions, the volume of submissions we receive makes it difficult to respond to everyone; as a general rule, if you haven’t heard from us within a week of submitting your post, it’s safe to assume we won’t move forward with publishing it.

Contribute to Towards Data Science

If you’re having an issue with our online form, please let us know via email (publication@towardsdatascience.com) so we can help you complete the process. Please do not email us an article that you have already sent via our form.

FAQ

Writing

• Do you publish only in English?
• What topics should I write about?
• Can I use an AI tool to help with my writing?
• How can I improve my writing skills?
• What featured image should I choose for my article?
• Can I write a post about my book, event, podcast, conference, etc.?
• Can I write about my company’s products?
• I’ve been asked to include a few sentences about the potential risks or ethical concerns of my topic/dataset. What do you expect?

Submission and Publishing

• Author Terms and Conditions of Use
• Privacy Policy
• My post has been refused, could you let me know why?

Data Sourcing

• How do I determine if a dataset license is okay?
• Can I use a dataset uploaded to a repository?
• Is data acquired through an API okay?
• What if there isn’t a copyright notice or license?
• What does an email asking for dataset permission include?

Advertising and Promotion

• Can I pay to get published?
• What kind of advertising or sponsored content do you allow?
• Can a company publish in TDS?

Others

• Can I translate a post to another language?
• Can I mention Towards Data Science in my Linkedin profile?
• How to contact you?

The post Write for Towards Data Science appeared first on Towards Data Science.

Since founding Towards Data Science in 2016, we’ve built the largest publication on Medium with a dedicated community of readers and contributors focused on data science, machine learning, and AI. Medium built a fantastic platform, and we wouldn’t have been able to reach our audience without its help.

As of Monday, February 3, 2025, Towards Data Science will become an independent publication, and we believe this transition will help us better serve our contributors and readers.

By moving off Medium we can maintain complete control over our editorial vision, protect the domain authority we’ve built, and guide our future growth on our own terms. 

Today, we’re launching our new WordPress site, though some parts of the migration are still a work in progress. We’re in close communication with our contributors to ensure a smooth transition. Despite the move, our mission remains unchanged: We’ll continue to provide the same in-depth coverage of data science, machine learning, and AI – only now on a platform where we can further support authors by utilizing exciting new promotional tools, advanced analytics, and dynamic community features that open up fresh opportunities for engagement and growth. 

Why Independence Makes Sense for TDS

We’ve long recognized that Medium provides valuable hosting, distribution and monetization tools – resources that have helped many writers and publishers (like us!) reach a wide audience. However,we believe our long term-term goals – as well as those of our readers and contributors – are best supported on an independent platform.

Here’s why we believe an independent platform is the right move:

Improved Accessibility for Readers – We believe in open access to all articles, not in paywalls. Moving to our own platform ensures that all of our content is available without charge to readers.

Ongoing Support for Authors – Our independent platform gives authors greater flexibility to promote their work. This means authors can amplify their voices and engage with new audiences.

Ownership of our Domain Authority – Operating independently allows us to preserve and grow Towards Data Science by not only retaining our search visibility, but also allowing us to experiment with novel content formats and create deeper connections with readers.

For current TDS authors, expect an email soon with details on how to join the new platform.

For new contributors, we’re opening up submissions and would love to feature your work on our new site. More details will follow in the coming days.

Content Rights Note

We want to be clear that we will only migrate content to our new site if we have the appropriate rights to do so in accordance with Towards Data Science’s terms and conditions. Your work remains under your control, and we’re committed to honoring any prior agreements regarding its use. Only content for which we have the necessary permissions will be transferred, ensuring that your intellectual property is respected throughout this transition.

If you have any questions or concerns about how your content will be handled during this migration, or wish to remove it from our new website, please don’t hesitate to reach out to our editorial team – you can email us here: publication@towardsdatascience.com. We’re here to provide any additional clarification you might need and to address any issues that arise.

How To Submit Your Work

We’re currently updating our submissions workflow, which we plan to launch in the near future. In the meantime, please fill out this form so we can already consider your article.

Contribution Submission Portal

FAQ: Transition for Authors & Contributors

We understand many TDS contributors have questions about how our move off Medium affects your work. Thank you for sticking with us through this change.

In the next few days, current and past authors will receive an email with instructions to set up a new account on our platform. This will let you update existing articles and submit new work for consideration.

Below are answers to some of the most pressing questions we’ve received so far, along with details about new features coming to Towards Data Science.

1. Why did Towards Data Science leave Medium?

Medium has been instrumental for our growth, but recent changes in Medium policies showed us that our priorities have diverged. By moving to our own platform, we gain full control over our editorial direction and policies. This independence allows us to provide free access to our articles, and better support you, our contributors, with a full set of publishing tools–including advanced analytics, enhanced social media capabilities, and more flexible content management–to help you reach a wider audience.

2. Why now?

At the end of January 2025, Medium began redirecting content from our custom domain, which immediately impacted our site traffic and search rankings. To protect our long-term vision and maintain the site’s hard-earned visibility, we fast-tracked a plan to launch an independent WordPress site.

3. When is the new platform launching?

Our new site launched on Monday, February 3, 2025. While we’re still refining some features on the backend, the site is fully operational with the paywall removed from all content. We’re working closely with contributors to ensure the transition is as smooth as possible. 

4. What do I need to do as a contributor during this transition?

For now, simply email your article to us in a PDF or Google Doc. We’ll handle the rest by importing your work into the new CMS and publishing it on the site. You don’t need to set up a new account right away–though current and past contributors will soon receive an email with instructions to create an account for managing and updating your content.

5. How will my existing content be handled?

We want to be clear: We only migrated content to our new site if we had the appropriate rights to do so in accordance with Towards Data Science’s terms and conditions. Your work remains under your control, and we’re committed to honoring any prior agreements regarding its use. Only content for which we have the necessary permissions will be transferred, ensuring that your intellectual property is respected throughout this transition.

If you have any questions or concerns about how your content will be handled during this migration, please don’t hesitate to reach out to our editorial team – you can email us here: publication@towardsdatascience.com.

We’re here to provide any additional clarification you might need and to address any issues that arise.

6. Can I continue to publish on Medium during this transition?

Yes, absolutely. Towards Data Science leaving Medium doesn’t mean you have to leave Medium. 

Please keep in mind that to publish on TDS, you’ll need to send us your article directly. We will no longer accept submissions through, or published on, Medium.

7. What are the benefits of publishing on the new TDS?

Moving to our own platform opens up several exciting opportunities for our contributors, including

• Unlimited Reach: Your articles will be completely free to read, free from Medium’s paywall. This means your work can reach a wider, more diverse audience.
• Advanced Promotional Tools: Our new site comes with enhanced features to empower authors to better promote their work. This flexibility will make it easier to use TDS to build your personal brand and connect with new readers. 
• Greater Editorial Control: We’re excited to launch new article formats that will allow authors greater control over how their content is presented to the reader.
• Better Visibility & Control: Content on TowardsDataScience.com will maintain its search rankings without being dependent on Medium’s policies.

8. Who do I contact if I have questions?

We’re here to help. If you have any additional questions or need further clarification, please email our editorial team at publication@towardsdatascience.com.

Your feedback is important as we finalize our migration to the new platform.

We hope these answers clarify our transition process. We’re thrilled about the future of Towards Data Science, and we’re committed to making this transition a smooth and positive experience for our authors and readers. Thank you for being a vital part of Towards Data Science as we take the publication to the next level!

– TDS Editorial Team

The post Towards Data Science is Launching as an Independent Publication appeared first on Towards Data Science.

The Variable is moving soon—sign up here to ensure you receive all future newsletters.

Our prolific authors delivered some excellent work this past month, channeling all the renewed energy and excitement we’ve come to expect from January on TDS. From career advice to core programming and data-processing tasks, our most-read and -shared articles in the past month cover the topics that data professionals care about the most as they plan their next move and aim to expand their skill set.

We invite you to explore this month’s must-reads with an open mind: from the ever-shifting terrain of job descriptions to the rise of small language models (alongside large ones), they tackle well-covered areas in Data Science and machine learning from a fresh, actionable, and pragmatic perspective. Let’s get started.

• How to Pick Between Data Science, Data Analytics, Data Engineering, ML Engineering, and SW Engineering"When the job titles sound so similar and the roles have a good amount of overlap" it can be difficult to choose the right path for your own interests and priorities as a data practitioner. Marina Wyss – Gratitude Driven‘s clear and detailed overview will help you make an informed decision.
• Your Company Needs Small Language ModelsIs it time to reassess the axiom that in AI, bigger is always better? Sergei Savvov makes a compelling case for the growing footprint of small language models in industry contexts, outlining the ways "they can reduce costs, improve accuracy, and maintain control of your data," and urges us to stay mindful of these models’ current limitations.
• The Large Language Model CourseFor anyone whose new year’s resolutions included expanding their knowledge of (and practical experience with) LLMs, Maxime Labonne‘s comprehensive course is the one-stop resource you’ll need to get started—it offers a well-structured curriculum that assumes no advanced knowledge, and comes full of recommended articles, tutorials, and tools.

• 5 Simple Projects to Start Today: A Learning Roadmap for Data EngineeringFor all the aspiring data engineers out there, Sarah Lea outlines a realistic, four-month plan that covers all the essentials. Most importantly, it guides you through several project ideas so that you not only grow your theoretical knowledge, but also get to practice the concepts and workflows you’re learning about.
• How to Build a Knowledge Graph in Minutes (And Make It Enterprise-Ready)Looking to apply a similar hands-on approach to knowledge graphs? After facing major setbacks in the past, Thuwarakesh Murallie shows how you can build one on your own by leveraging the power of LLMs.
• Top 12 Skills Data Scientists Need to Succeed in 2025"Many things are changing, but other things are not. Understanding which changes require your attention is the key to success." Benjamin Bodner breaks down the skills that remain essential for data professionals amid the disruptive pressure of AI tools.
• Deep Dive into Multithreading, Multiprocessing, and AsyncioWhether you’re taking your first steps in Python or are already a seasoned programmer, there’s always more to learn; Clara Chong‘s recent post zooms in on concurrency models—approaches to handle multiple tasks simultaneously—and unpacks the stakes of choosing the right one, depending on your project’s needs.
• How to Run Jupyter Notebooks and Generate HTML Reports with Python ScriptsFor another Python-focused tutorial that foregrounds the ability to streamline tedious workflows with code, don’t miss Amanda Iglesias Moreno‘s guide to automating Jupyter Notebook execution and report generation (with a helpful detour through synthetic-data creation).
• My Experience Switching From Power BI to Looker (as a Senior Data Analyst)It’s often difficult to assess the tradeoffs and gains you’ll be making by adopting a new tool without first trying different options—or hearing firsthand accounts from professionals in a similar situation to yours. Tomas Jancovic (It’s AI Thomas) walks us through the journey of moving to Looker for his data analytics workflows, and presents a balanced, concrete account of its pros and cons.
• Three Important Pandas Functions You Need to KnowNew models and generative-AI apps come and go, but Pandas is still here with us, playing an important role in data scientists’ day-to-day work. Jiayan Yin recently shared a helpful guide for early-stage learners, zooming in on three essential functions that you’ll likely turn to again and again as you process and analyze datasets.

Our latest cohort of new authors

Every month, we’re thrilled to see a fresh group of authors join TDS, each sharing their own unique voice, knowledge, and experience with our community. If you’re looking for new writers to explore and follow, just browse the work of our latest additions from the past couple of months, including Ramsha Ali, Derick Ruiz, Dr. Marcel Müller, Rodrigo M Carrillo Larco, MD, PhD, Ilona Hetsevich, Federico Zabeo, Vladyslav Fliahin, Jérôme DIAZ, Mandeep Kular, Glenn Kong, Vladimir Kukushkin, Viktor Malyi, Ruben Broekx, Iqbal Hamdi, Richa Gadgil, Piotr Gruszecki, Jonathan Fürst, Sirine Bhouri, Kyoosik Kim, Sunghyun Ahn, Afjal Chowdhury, Tim Wibiral, Kunal Santosh Sawant, Aman Agrawal, Abdelkader HASSINE, Florian Trautweiler, Mohammed AbuSadeh, Loic Merckel, Lukasz Gatarek, Zombor Varnagy-Toth, Marc Matterson, Manelle Nouar, Paula LC, Shitanshu Bhushan, Matthew Senick, Lewis James | Data Science, Clara Chong, Bilal Ahmed, Pavel Krautsou, Erol Çıtak, Cristovao Cordeiro, Vladimir Zhyvov, Yuval Gorchover, Zach Flynn, Allon Korem | CEO, Bell Statistics, Tony Albanese, Sandra E.G., Miguel Cardona Polo, James Thorn, Vineet Upadhya, Kaushik Rajan, Mahmoud Abdelaziz, PhD, Benjamin Assel, Shirley Li, Marina Wyss – Gratitude Driven, Michal Davidson, Rémy Garnier, Uladzimir Yancharuk, David Lindelöf, Ricardo Ribas, Hunjae Timothy Lee, Ashley Peacock, Rohit Ramaprasad, Alejandro Alvarez Pérez, David Martin, Ben Tengelsen, César Ortega Quintero, Jaemin Han, Max Surkiz, Massimo Capobianco, Tobias Cabanski, Jimin Kang, Felix Schmidt, Paolo Molignini, PhD, Sayali Kulkarni, Alan Nekhom, and Chris Lettieri, among others.

Thank you for supporting the work of our authors! We love publishing articles from new authors, so if you’ve recently written an interesting project walkthrough, tutorial, or theoretical reflection on any of our core topics, don’t hesitate to share it with us.

Until the next Variable,

TDS Team

The post Data Roles, Small Language Models, Knowledge Graphs, and More: Our January Must-Reads appeared first on Towards Data Science.

Whether you’re an independent practitioner or part of a larger team trying to make sense of this emerging technology, you’ll find concrete and actionable insights in the lineup of articles we’ve selected this week. They each tackle the nuts and bolts of building AI apps and leveraging their potential for well-defined goals, while avoiding common pain points.

While these posts zoom in on specific topics and business problems, they all offer a pragmatic, accessible approach, making them useful for readers across a wide spectrum of backgrounds and experience levels. Let’s dive in.

• What Did I Learn from Building LLM Applications in 2024? – Part 2So you’ve built a prototype of an LLM-based app that looks promising. What’s next? Satwiki De offers a comprehensive roadmap based on the takeaways she’s accumulated over a year of experimentation—the main thrust being to "develop your AI-enabled application keeping the business objectives in mind."
• Why Generative-AI Apps’ Quality Often Sucks and What to Do About itHaving observed countless enterprise AI initiatives deliver underwhelming results (if not fail altogether) Dr. Marcel Müller focuses on the importance of robust assessment, and shows "how we can qualitatively and quantitatively evaluate generative AI applications in the context of concrete business processes."
• Designing, Building & Deploying an AI Chat App from Scratch (Part 1)If you’re ready to roll up your sleeves (real or proverbial) and start implementing an AI chat app, Joris Baan‘s patient guide is a great resource to help you stay on the right track. Part 1 outlines the microservices architecture and local development needs you’ll want to think about, while part 2 moves on to a detailed discussion of cloud deployment and scaling.

• Learn to Build Advanced AI Image ApplicationsMoving beyond chatbots, Ida Silfverskiöld explores the possibilities that visual generative-AI tools open up for real-world businesses and professionals—in this case, an interior design app based entirely on open-source models and frameworks.
• How to Build an AI Agent for Data Analytics Without Writing SQLWhy not harness the power of AI to streamline common data-analytics workflows that would typically require numerous SQL queries? Chengzhi Zhao aims to accomplish just that with the aid of an AI agent, and shows how you can build one yourself with LangChain and DuckDB.
• AI-Powered Information Extraction and MatchmakingFor another promising use case where an AI app can help you and your team save time and become more efficient, don’t miss Umair Ali Khan‘s patient tutorial on building AI-based match-making tools (in this case, helping job seekers find positions that fit their skills and interests).

Branching out into the world beyond AI apps, we’ve selected a few more recommended reads we thought you’d enjoy—from a beginner-friendly intro to LLMs to an in-depth analysis of data strategies.

• If your less tech-savvy colleagues could use a clear and accessible primer on what LLMs are and how they work, just send them Carolina Bento‘s top-notch introduction.
• For a more advanced exploration of LLMs’ inner workings, head right over to Jaemin Han‘s fascinating look at models’ shortcomings in generating and interpreting ASCII art—and the surprising security risks the latter might suggest.
• Quick and focused, Clara Chong‘s guide to variable scoping explains why this particular aspect of your code can have far-reaching consequences for your Data Science workflows.
• What rules should we have in place to protect humans from unknowingly interacting with AI profiles on social platforms? James Barney unpacks the fallout from the recent controversy around Meta’s (now-suspended) experiment.
• The rise of code-generating chatbots leads Murtaza Ali to wonder if the age of human-written programming tutorials is coming to an end. (Spoiler alert / sigh of relief: not just yet!)
• If you’d like to make sense of the gap you observe between between predictions and real-world outcomes, Hennie de Harder‘s new post walks us through the basics of prediction probabilities, calibration, and how to interpret these numbers in a practical context.
• In the mood for a deep dive? Jens Linden, PhD just shared the latest installment in his Demystify Data Strategy series, focusing this time on the common pitfalls of data and AI strategies—and how organizations can avoid them.
• To round out this week’s lineup, we invite you to learn about MicroPython and its potential relevance for data scientists; Sarah Lea‘s concise guide will get you up to speed on all the essential details.

Thank you for supporting the work of our authors! As we mentioned above, we love publishing articles from new authors, so if you’ve recently written an interesting project walkthrough, tutorial, or theoretical reflection on any of our core topics, don’t hesitate to share it with us.

Until the next Variable,

TDS Team

The post Building Successful AI Apps: The Dos and Don’ts appeared first on Towards Data Science.

Feeling inspired to write your first TDS post? We’re always open to contributions from new authors.

Buzzwords and trends come and go, but the core task of telling compelling stories with data remains one of the main pillars in data scientists’ daily workflow. For practitioners who’d like to up their visualization game, this week we’re highlighting some of our best recent articles on creating powerful, effective, and sleek deliverables.

Our selection tackles the topic from multiple angles, so whether you’re interested in chart optimization, geospatial aids, or interactive dashboards, we’re sure you’ll find something here to inspire you and help you expand your current skill set. Happy tinkering!

• How to Make Extremely Beautiful Charts with PythonWhy settle for "meh…" when "wow!" is within reach? Ari Joury, PhD challenges us to stop settling for mediocre visualizations, shows how a few smart and focused tweaks can transform your average chart into something you can truly be proud of, and offers more than a few concrete examples (and code snippets) to make these suggestions both concrete and actionable.
• 5 Essential Tips to Build Business Dashboards Stakeholders Love"You might be wondering, isn’t dashboarding just creating a bunch of charts? Well, Yes and No." Yu Dong zooms in on an often-undervalued aspect of data professionals’ work, and explains why it’s time to reevaluate and streamline your approach to dashboard creation.
• Awesome Plotly with Code Series (Part 8): How to Balance Dominant Bar Chart CategoriesFor the past few months, Jose Parreño‘s excellent series has addressed some of the main pain points of designing functional, easy-to-digest charts. His latest installment takes a close, pragmatic look at a common scenario where one category threatens to throw an entire bar chart off-balance.

• Step-by-Step Guide for Building Bump Charts in PlotlyReady to move on from bar charts into more advanced and custom formats? Don’t miss Amanda Iglesias Moreno‘s Plotly-based tutorial, which introduces a complete workflow for creating a bump chart, a more specialized visualization that is "designed to explore changes in a ranking over time" and allows us to "quickly identify trends and detect elements at the top or bottom of the ranking."
• Easy Map Boundary Extraction with GeoPandasWorking with geospatial data can be very rewarding—not to mention essential in many industries—but it can also get tricky and occasionally unwieldy. Lee Vaughan‘s latest Python guide brings clarity and practicality to a very common use case: extracting, measuring, and plotting country borders.

A new year often brings with it a rush of excellent new writing, and so far 2025 has not disappointed on that front. Here are several recent standouts on a wide range of topics, from hands-on AI projects to the history of GPT models.

• What would it take for a supervised-learning model to solve a Rubik’s Cube? Daniel Warfield‘s latest deep dive walks us through a fun and enlightening project.
• As more and more AI-generated content floods contemporary media and online spaces, Stephanie Kirmer wonders how we should assess its cultural impact—and navigate its (many) pitfalls.
• For a solid understanding of cutting-edge LLMs, a trip down memory lane (all the way back to the prehistoric era of 2019–2020) might be a good idea—and Shirley Li‘s thorough and accessible retrospective of GPT-2 and GPT-3 offers us precisely that.
• If you’re in the mood for something more hands-on this week, we highly recommend Maeda Hanafi, PhD‘s new guide to customizing your fine-tuning code with Hugging Face’s Transformers library.
• Are we approaching the point where AI might be able to reliably determine if something is funny? Vineet Upadhya‘s latest article reviews recent research into this fascinating topic, including a new framework for computational humor detection.
• Establishing causality is a common goal for data practitioners, but that doesn’t make it any less difficult to pull off. For his debut TDS post, Rémy Garnier presents a versatile method that you can apply across a wide range of use cases.
• After sharing a well-received list of AI projects you can experiment with over a weekend, Shaw Talebi fleshes out the details of one of his proposed ideas: an AI-powered resume optimizer.

Thank you for supporting the work of our authors! As we mentioned above, we love publishing articles from new authors, so if you’ve recently written an interesting project walkthrough, tutorial, or theoretical reflection on any of our core topics, don’t hesitate to share it with us.

Until the next Variable,

TDS Team

The post Charts, Dashboards, Maps, and More: Data Visualization in the Spotlight appeared first on Towards Data Science.

Feeling inspired to write your first TDS post? We’re always open to contributions from new authors.

As we all settle into the sometimes hectic rhythm of a new year, we hope you’ve been enjoying the excitement of kicking off projects, learning about new topics, and exploring your next career moves. We’re definitely seeing a flurry of activity among our authors—both longstanding contributors and recent additions—and are thrilled to share all the great work they’ve been cooking up over the holidays.

Our lineup of top-notch reads this week has a distinctly actionable, hands-on flavor to it—after all, what better way to harness all this energy than by tinkering with some datasets, models, and code? Whether you’re interested in learning more about cutting-edge evaluation methods or building agentic-AI tools, we’ve got you covered with a diverse selection of tutorials and practical overviews. Ready to dive in?

• Paradigm Shifts of Eval in the Age of LLMs Is it time to reevaluate the way we approach evaluations? Lili Jiang believes it is: "I’ve come to recognize that LLMs requires some subtle, conceptually simple, yet important changes in the way we think about evaluation." Her latest article offers high-level insights into what a new paradigm might look like.

• The Next Frontier in LLM Accuracy Staying thematically close to LLM optimization, Mariya Mansurova‘s new deep dive unpacks in great detail several methods we can use to increase models’ accuracy, and zooms in on advanced fine-tuning techniques.

• How to Build a Graph RAG App Ready to roll up your sleeves and dig deep into some code? Steve Hedden‘s thorough tutorial on creating your first graph RAG app is a great option for anyone who’s interested in this trending topic but needs guidance and context to ensure they’re starting off on the right foot.

• Multi-Agentic RAG with Hugging Face Code Agents Agent-based systems gained enormous steam (and buzz) last year, and it doesn’t seem like that’s about to change in 2025. Curious to learn more about them? Gabriele Sgroi, PhD‘s patient, step-by-step guide may be long, but it remains accessible and clear as it outlines the process of leveraging a "small" LLM to power a multi-agentic system—and produce good results, even on consumer-grade hardware.

• Demand Forecasting with Darts: A TutorialLLMs may be grabbing much of our collective attention these days, but business-focused workflows remain the bread and butter of industry data scientists. Sandra E.G.‘s debut TDS article provides a robust, hands-on introduction to one such essential task: demand forecasting in the context of retail sales.

• Distributed Parallel Computing Made Easy with Ray It’s crucial for data and ML practitioners to experiment with new tools and frameworks, as seemingly small improvements can accumulate into major cost and efficiency benefits. Betty LD walks us through her recent foray into the AI-focused Ray library for distributed data processing, and demonstrates its power through the use case of scalable offline batch inference.

If you’re ready to branch out into other topics this week, we’re here to help—whether your interests lie at the intersection of music and AI, quantum computing, or linear algebra (among others), we hope you explore some of these excellent articles:

• Presenting her team’s cutting-edge research, Tula Masterman explains how we can leverage the hidden state from an intermediate Transformer layer for efficient and robust content safety and prompt injection classification.
• Music-centered AI tools continue to make strides; Max Hilsdorf devotes his latest exploration of the topic to mono-to-stereo upmixing, a technique for enriching or improving our music-listening experience.
• With seven first-author publications under her belt, Malak Sadek is well-positioned to offer concrete insights for other researchers who’d like to grow their publishing footprint.
• Continuing his ongoing series on key linear algebra concepts, Rohit Pandey recently shared a new, comprehensive explainer that looks at the inner workings of orthonormal matrices.
• Quantum computing has been a hot topic for some time now, though discussions of its promise can sometimes feel almost sci-fi-adjacent. Here to help us make sense of the field and where things stand at the moment is Sara A. Metwalli, whose primer comes right as the UN has declared 2025 the international year of quantum science and technology.
• While we’re on the topic, let’s round out this week’s selection with Benjamin Assel‘s nuanced discussion of error-rate measurement in IBM quantum processors, which also includes code examples, using Qiskit.

Thank you for supporting the work of our authors! As we mentioned above, we love publishing articles from new authors, so if you’ve recently written an interesting project walkthrough, tutorial, or theoretical reflection on any of our core topics, don’t hesitate to share it with us.

Until the next Variable,

TDS Team

The post LLM Evaluation, Parallel Computing, Demand Forecasting, and Other Hands-On Data Science Approaches appeared first on Towards Data Science.

Feeling inspired to write your first TDS post? We’re always open to contributions from new authors.

Happy new year! Welcome back to the Variable!

The ink has barely dried on our 2024 highlights roundup (it’s never too late to browse it, of course), and here we are, ready to dive headfirst into a fresh year of learning, growth, and exploration.

We have a cherished tradition of devoting the first edition of the year to our most inspiring—and accessible—resources for early-stage Data Science and machine learning professionals (we really do!). We continue it this year with a selection of top-notch recent articles geared at beginner-level learners and job seekers. For the rest of our readers, we’re thrilled to kick things off with a trio of excellent posts from industry veterans who reflect on the current state of data science and AI, and share their opinionated, bold predictions for what the year ahead might look like. Let’s get started!

2025: Ready, Set, Go!

• Top 10 Data & AI Trends for 2025As the co-founder and CEO of Monte Carlo, few people are as well-positioned as Barr Moses to observe industry shifts and emerging trends—so her annual forecast should definitely be on your to-read list.
• 2024 in Review: What I Got Right, Where I Was Wrong, and Bolder Predictions for 2025After a year dominated by RAG, AI agents, and knowledge graphs, what tools and workflows should we keep track of in the coming year? Leonie Monigatti shares a thorough recap of 2024—and a sneak peek at her well-tuned crystal ball for the year to come.
• Ten Predictions for Data Science and AI in 2025From Jason Tamara Widjaja, here’s another set of insightful reflections: "My predictions for 2025 attempt to provide a view of the tensions of AI, taking an unpopular but balanced view as someone whose work depends not on selling AI, but on implementing AI well – and living through the consequences of our decisions."

Data science and machine learning, step by step by step

• The Essential Guide to R and Python Libraries for Data VisualizationWith or without AI, charts and plots aren’t going anywhere anytime soon. Sarah Lea maps out the key libraries in which current and aspiring data scientists should gain fluency.
• Roadmap to Becoming a Data Scientist, Part 2: Software EngineeringProgramming isn’t going anywhere in 2025, either. Vyacheslav Efimov‘s guide outlines the coding essentials that will lead you to data science success.
• Missing Data in Time-Series: Machine Learning TechniquesOne constant trait of real-world data: it’s messy! Learn how to navigate the chaos by following along Sara Nóbrega‘s primer on handling missing data.
• Causality – Mental Hygiene for Data ScienceTaking a few steps back from the more nitty-gritty aspects of data science work, Eyal Kazin‘s recent deep dive constitutes a "gentle intro" to the intricate art of detecting, interpreting, and applying causality.
• Master Machine Learning: 4 Classification Models Made SimpleFor anyone who enjoys structure and clarity above all else, Leo Anello‘s (extremely) thorough, 15-step tutorial on classification models would be a perfect starting point from which to expand your ML know-how.
• 2024 Survival Guide for Machine Learning Engineer InterviewsWhether you’re already applying for your first MLE job or contemplating it as one of your goals for the year, don’t miss Mengliu Zhao‘s "survival guide," aimed specifically at junior-level practitioners.
• Machine Learning Basics I Look for in Data Scientist InterviewsTackling the occasionally opaque hiring process from the other end of the table, Farzad Nobar created a helpful resource to help job applicants zoom in on the topics that really matter to employers.
• 100 Years of (eXplainable) AIBeyond the whats and hows of day-to-day work, there are also the whys: why did this model produce these outputs? Sofya Lipnitskaya‘s explainer unpacks the history of AI explainability in the context of the recent rise of LLMs.
• How to Build a General-Purpose LLM AgentTo end on a more hands-on note—and to satisfy the curiosity of all of you who’ve heard the buzz around AI agents—we highly recommend Maya Murad‘s step-by-step guide, which can form "the groundwork for designing your own custom agentic architecture" down the line.

Thank you for supporting the work of our authors! As we mentioned above, we love publishing articles from new authors; if contributing to TDS in 2025 is one of your new year’s resolutions—or even if you’ve just recently written an interesting project walkthrough, tutorial, or theoretical reflection on any of our core topics—don’t hesitate to share it with us.

Until the next Variable,

TDS Team

The post Start a New Year of Learning on the Right Foot appeared first on Towards Data Science.

Feeling inspired to write your first TDS post before the end of 2024? We’re always open to contributions from new authors.

And just like that, 2024 is (almost) in the books. It was a year of exciting transitions – both for the TDS team and, in many meaningful ways, for the data science, machine learning, and AI communities at large. We’d like to thank all of you—readers, authors, and followers—for your support, and for keeping us busy and engaged with your excellent contributions and comments.

Unlike in 2023, when a single event (ChatGPT’s launch just weeks before the beginning of the year) stopped everyone in their tracks and shaped conversations for months on end, this year we experienced a more cumulative and fragmented sense of transformation. Practitioners across industry and academia experimented with new tools and worked hard to find innovative ways to benefit from the rapid rise of LLMs; at the same time, they also had to navigate a challenging job market and a world where AI’s footprint inches ever closer to their own everyday workflows.

To help you make sense of these developments, we published more than 3,500 articles this past year, including hundreds from first-time contributors. Our authors have an incredible knack for injecting their unique perspective into any topic they cover—from big questions and timely topics to more focused technical challenges—and we’re proud of every post we published in 2024.

Within this massive creative output, some articles manage to resonate particularly well with our readers, and we’re dedicating our final Variable edition to these: our most-read, -discussed, and -shared posts of the year. As you might expect, they cover a lot of ground, so we’ve decided to arrange them following the major themes we’ve detected this year: learning and building from scratch, RAG and AI agents, career growth, and breakthroughs and innovation.

We hope you enjoy exploring our 2024 highlights, and we wish you a relaxing end of the year – see you in January!

Learning and Building from Scratch

The most reliably popular type of TDS post is the one that teaches readers how to do or study something interesting and productive on their own, and with minimal prerequisites. This year is no exception—our three most-read articles of 2024 fall under this category.

• 5 AI Projects You Can Build This Weekend (with Python)From beginner-friendly to advanced project ideas, Shaw Talebi demonstrates that anyone can get hands-on with AI.
• Understanding LLMs from Scratch Using Middle School MathHow do LLMs work? Rohit Patel offered one of the most accessible and engaging explainers you’ll ever find on this topic.
• How to Learn AI on Your Own (A Self-Study Guide)For the self-starters out there, Thu Vu put together a streamlined roadmap for studying the fundamental building blocks of AI.
• The Math Behind Neural NetworksTo understand neural networks, "the backbone of modern AI," Cristian Leo guides us deep into their underlying mathematical principles.
• Text Embeddings: Comprehensive GuideEmbeddings make the magic of LLMs possible, and Mariya Mansurova‘s thorough introduction makes it clear how and why they’ve become so important.
• How I Studied LLMs in Two Weeks: A Comprehensive RoadmapAnother excellent learning resource came to us from Hesam Sheikh, who walked us through an intensive—but accessible— curriculum to master the basics (and then some) of large language models.

RAG and AI Agents

Once the initial excitement surrounding LLMs settled (a bit), data and ML professionals realized that these powerful models aren’t all that useful out of the box. Retrieval-augmented generation and agentic AI rose to prominence in the past year as the two leading approaches that bridge the gap between the models’ potential and real-world value; they also ended up being our most covered technical topics in recent months.

• Intro to LLM Agents with LangChain: When RAG Is Not EnoughBack in March—and quite ahead of the curve—Alex Honchar published the definitive beginners’ guide to working with agents.
• Using LangChain ReAct Agents for Answering Multi-Hop Questions in RAG SystemsShowing us how agents and RAG can complement each other, Dr. Varshita Sher‘s tutorial addresses the common need of answering complex queries on internal documents.
• 17 (Advanced) RAG Techniques to Turn Your LLM App Prototype into a Production-Ready SolutionBuilding a rudimentary RAG pipeline is one thing; optimizing it so that it can actually work in a business context is another. Dominik Polzer put together a comprehensive guide to the methods you can leverage to achieving that lofty goal.
• 12 RAG Pain Points and Proposed SolutionsOn a similar troubleshooting beat, Wenqi Glantz outlines a dozen streamlined approaches for tackling some of the most common challenges practitioners face when implementing RAG.
• Choosing Between LLM Agent FrameworksIt can be tough to make informed choices in an ecosystem where both major and emerging players release new tools every day. Aparna Dhinakaran is here to help with sharp insights on the tradeoffs to keep in mind.

Career Growth

Data Science and machine learning career paths continue to evolve, and the need to adapt to this changing terrain can generate nontrivial amounts of stress for many professionals, whether they’re deep into their career or are just starting out. We love publishing personal reflections on this topic when they also offer readers pragmatic advice—here are four that stood out to us (and to our readers).

• What 10 Years at Uber, Meta, and Startups Taught Me About Data AnalyticsFrom the importance of storytelling and business acumen to the limitations of metrics, Torsten Walbaum generously consolidated lessons based on a decade of work into actionable insights.
• What I Learned in my First 3 Months as a Freelance Data ScientistCareer switches are always tricky, and moving away from the structure of working at a company to the world of self-employment comes with its own set of challenges—and, as CJ Sullivan shows, with great opportunities for learning and growth.
• How I Became A Data Scientist – No CS Degree, No BootcampFor anyone just taking their first steps in the field, Egor Howell‘s candid account of his path into data science is a must-read.
• I Spent $96k To Become a Data Scientist. Here Are 5 Crucial Lessons All Beginners Must KnowOffering a different perspective on entering the discipline, Khouloud El Alami offers practical tips on managing your data science education so that you set yourself on the right path.

Breakthroughs and Innovation

Staying up-to-date with cutting-edge research and new tools can feel overwhelming at times, which is why we have a particular soft spot for top-notch paper walkthroughs and primers on emerging libraries and models. Here are three such articles that particularly resonated with our audience.

• A New Coefficient of Correlation"What if you were told there exists a new way to measure the relationship between two variables just like correlation except possibly better"? So starts Tim Sumner‘s explainer on a groundbreaking 2020 paper.
• Intro to DSPy: Goodbye Prompting, Hello Programming!In another exciting year for open-source tools, one of the standout new arrivals was DSPy, which aims to open up LLMs for programmers and make it easier to build modular AI solutions. Leonie Monigatti‘s hands-on introduction is the perfect place to start exploring its possibilities.
• Kolmogorov-Arnold Networks: The Latest Advance in Neural Networks, Simply ExplainedKANs, "promising alternatives of Multi-Layer Perceptrons (MLPs)," made a splashy entrance last spring; Theo Wolf made their ramifications and potential benefits for ML practitioners evident with this accessible primer.

Thank you for supporting the work of our authors in 2024! If writing for TDS is one of your goals for 2025, why not get started now? Don’t hesitate to share your work with us.

Until the next Variable, coming your way in the first week of January,

TDS Team

The post 2024 Highlights: The AI and Data Science Articles That Made a Splash appeared first on Towards Data Science.