Hitchhikers-guide - The Hitchhiker's Guide to Data Science for Social Good

Overview

Welcome to the Hitchhiker's Guide to Data Science for Social Good.

What is the Data Science for Social Good Fellowship?

The Data Science for Social Good Fellowship (DSSG) is a hands-on and project-based summer program that launched in 2013 at the University of Chicago and has now expanded to multiple locations globally. It brings data science fellows, typically graduate students, from across the world to work on machine learning, artificial intelligence, and data science projects that have a social impact. From a pool of typically over 800 applicants, 20-40 fellows are selected from diverse computational and quantitative disciplines including computer science, statistics, math, engineering, psychology, sociology, economics, and public policy.

The fellows work in small, cross-disciplinary teams on social good projects spanning education, health, energy, transportation, criminal justice, social services, economic development and international development in collaboration with global government agencies and non-profits. This work is done under close and hands-on mentorship from full-time, dedicated data science mentors as well as dedicated project managers, with industry experience. The result is highly trained fellows, improved data science capacity of the social good organization, and a high quality data science project that is ready for field trial and implementation at the end of the program.

In addition to hands-on project-based training, the summer program also consists of workshops, tutorials, and ethics discussion groups based on our data science for social good curriculum designed to train the fellows in doing practical data science and artificial intelligence for social impact.

Who is this guide for?

The primary audience for this guide is the set of fellows coming to DSSG but we want everything we create to be open and accessible to larger world. We hope this is useful to people beyond the summer fellows coming to DSSG.

If you are applying to the program or have been accepted as a fellow, check out the manual to see how you can prepare before arriving, what orientation and training will cover, and what to expect from the summer.

If you are interested in learning at home, check out the tutorials and teach-outs developed by our staff and fellows throughout the summer, and to suggest or contribute additional resources.

*Another one of our goals is to encourage collaborations. Anyone interested in doing this type of work, or starting a DSSG program, to build on what we've learned by using and contributing to these resources.

What is in this guide?

Our number one priority at DSSG is to train fellows to do data science for social good work. This curriculum includes many things you'd find in a data science course or bootcamp, but with an emphasis on solving problems with social impact, integrating data science with the social sciences, discussing ethical implications of the work, as well as privacy, and confidentiality issues.

We have spent many (sort of) early mornings waxing existential over Dunkin' Donuts while trying to define what makes a "data scientist for social good," that enigmatic breed combining one part data scientist, one part consultant, one part educator, and one part bleeding heart idealist. We've come to a rough working definition in the form of the skills and knowledge one would need, which we categorize as follows:

  • Programming, because you'll need to tell your computer what to do, usually by writing code.
  • Computer science, because you'll need to understand how your data is - and should be - structured, as well as the algorithms you use to analyze it.
  • Math and stats, because everything else in life is just applied math, and numerical results are meaningless without some measure of uncertainty.
  • Machine learning, because you'll want to build predictive or descriptive models that can learn, evolve, and improve over time.
  • Social science, because you'll need to know how to design experiments to validate your models in the field, and to understand when correlation can plausibly suggest causation, and sometimes even do causal inference.
  • Problem and Project Scoping, because you'll need to be able to go from a vague and fuzzy project description to a problem you can solve, understand the goals of the project, the interventions you are informing, the data you have and need, and the analysis that needs to be done.
  • Project management, to make progress as a team, to work effectively with your project partner, and work with a team to make that useful solution actually happen.
  • Privacy and security, because data is people and needs to be kept secure and confidential.
  • Ethics, fairness, bias, and transparency, because your work has the potential to be misused or have a negative impact on people's lives, so you have to consider the biases in your data and analyses, the ethical and fairness implications, and how to make your work interpretable and transparent to the users and to the people impacted by it.
  • Communications, because you'll need to be able to tell the story of why what you're doing matters and the methods you're using to a broad audience.
  • Social issues, because you're doing this work to help people, and you don't live or work in a vacuum, so you need to understand the context and history surrounding the people, places and issues you want to impact.

All material is licensed under CC-BY 4.0 License: CC BY 4.0

Table of Contents

The links below will help you find things quickly.

DSSG Manual

Summer Overview

This sections covers general information on projects, working with partners, presentations, orientation information, and the following schedules:

Conduct, Culture, and Communications

This section details the DSSG anti-harassment policy, goals of the fellowship, what we hope fellows get out of the experience, the expectations of the fellows, and the DSSG environment. A slideshow version of this can also be found here.

Curriculum

This section details the various topics we will be covering throughout the summer. This includes:

Wiki

In the wiki, you will find a bunch of helpful information and instructions that people have found helpful along the way. It covers topics like:

  • Accessing S3 from the command line
  • Creating an alias to make Python3 your default (rather than python2)
  • Installing RStudio on your EC2
  • Killing your query
  • Creating a custom jupyter setup
  • Mounting box from ubuntu
  • Pretty Print psql and less output
  • Remotely editing text files in your favorite text editor
  • SQL Server to Postgres
  • Using rpy2
  • VNC Viewer
Owner
Data Science for Social Good
Data Science for Social Good
Python implementation for Active Directory certificate abuse

Certipy is a Python tool to enumerate and abuse misconfigurations in Active Directory Certificate Services (AD CS). Based on the C# variant Ce

Oliver Lyak 1.3k Jan 09, 2023
A server shell for you to play with Powered by Django + Nginx + Postgres + Bootstrap + Celery.

A server shell for you to play with Powered by Django + Nginx + Postgres + Bootstrap + Celery.

Mengting Song 1 Jan 10, 2022
Simple project to assist in tracking/logging my working hours

Fill working hours Basic script to assist in the logging/tracking of my working hours How it works Create a file called projects.json in this director

Robin Kennedy-Reid 2 Oct 31, 2022
A not exist cat image generator python package

A not exist cat image generator python package

Fayas Noushad 2 Dec 03, 2021
This interactive script demonstrates the Menezes-Vanstone-EC-Cryptosystem

Menezes-Vanstone-EC-Cryptosystem This interactive script demonstrates the Meneze

Nishaant Goswamy 1 Jan 02, 2022
The repository for AnyMacro: a Fusion360 Add-In

AnyMacro AnyMacro is an Autodesk® Fusion 360™ add-in for chaining multiple commands in a row to form Macros. Macros are created from a set of commands

1 Jan 07, 2022
Open-source data observability for modern data teams

Use cases Monitor your data warehouse in minutes: Data anomalies monitoring as dbt tests Data lineage made simple, reliable, and automated dbt operati

889 Jan 01, 2023
A Python package that provides physical constants.

PhysConsts A Python package that provides physical constants. The code is being developed by Marc van der Sluys of the department of Astrophysics at t

Marc van der Sluys 1 Jan 05, 2022
switching computer? changing your setup? You need to automate the download of your current setup? This is the right tool for you :incoming_envelope:

🔮 setup_shift(SS.py) switching computer? changing your setup? You need to automate the download of your current setup? This is the right tool for you

Mohamed Elfaleh 15 Aug 26, 2022
The Python Fuzzer that the world deserves 🐍

pip3 install frelatage Current release : 0.0.2 The Python Fuzzer that the world deserves Installation | How it works | Features | Use Frelatage | Conf

Rog3r 219 Dec 21, 2022
RestMapper takes the pain out of integrating with RESTful APIs.

python-restmapper RestMapper takes the pain out of integrating with RESTful APIs. It removes all of the complexity with writing API-specific code, and

Lionheart Software 8 Oct 31, 2020
Aesthetic NFT Generator

A E S T H E T I C Dependencies Pillow numpy OpenCV You can use pip to install any missing dependencies. Basic Usage Vaporwave artwork can be generated

Mentor Elezi 4 Mar 13, 2022
A Blender addon for VSE that auto-adjusts video strip's length, if speed effect is applied.

Blender VSE Speed Adjust Addon When using Video Sequence Editor in Blender, the speed effect strip doesn't auto-adjusts clip length when changing its

Arpit Srivastava 2 Jan 18, 2022
Sodium is a general purpose programming language which is instruction-oriented (a new programming concept that we are developing and devising) [Still developing...]

Sodium Programming Language Sodium is a general purpose programming language which is instruction-oriented (a new programming concept that we are deve

Instruction Oriented Programming 22 Jan 11, 2022
A tool to assist in code raiding in rust

Kodelock a tool to assist in code raiding in rust This tool is designed to be used on a second monitor. This tools will allow you to see a easily read

3 Oct 27, 2022
Like Docker, but for Squeak. You know, for kids.

Squeaker Like Docker, but for Smalltalk images. You know, for kids. It's a small program that helps in automated derivation of configured Smalltalk im

Tony Garnock-Jones 14 Sep 11, 2022
This is an implementation of NeuronJ work with python.

NeuronJ This is an implementation of NeuronJ work with python. NeuronJ is a plug-in for ImageJ that allows you to create and edit neurons masks. Image

Mohammad Mahdi Samei 3 Aug 28, 2022
A Bot Which Can generate Random Account Based On Your Hits.

AccountGenBot This Bot Can Generate Account With Hits You Save (Randomly) Keyfeatures Join To Use Support Limit Account Generation Using Sql Customiza

DevsExpo 30 Oct 21, 2022
VacationCycleLogicBackEnd - Vacation Cycle Logic BackEnd With Python

Vacation Cycle Logic BackEnd Getting Started Existing virtualenv If your project

Mohamed Gamal 0 Jan 03, 2022
A Company Management System For Python

campany-management Getting started To make it easy for you to get started with GitLab, here's a list of recommended next steps. Already a pro? Just ed

hatice akpınar 3 Aug 29, 2022