Proyecto - Análisis de texto de eventos históricos

Overview

Acceder al código desde Google Colab para poder ver de manera adecuada todas las visualizaciones y poder interactuar con ellas.

Link de acceso: https://colab.research.google.com/drive/1XqDm6szrNG8ZdH37EVITPCSw7BDZFFQ5?usp=sharing

Corto video explicativo: https://youtu.be/ZDPXc56jOj4

Proyecto Big Data - Análisis de texto de eventos históricos

Declaración del conjunto de datos

Contamos con un dataset en formato JSON proveniente del repositorio 'awesome-json-datasets' en la sección 'Historical Events' sobre eventos históricos (disponible en: https://github.com/jdorfman/awesome-json-datasets). Este dataset cuenta con información desde el año 299 A.C. hasta el año 2013. Recopila sucesos importantes en el mundo a lo largo de este periodo señalado.

La estrucutra de cada recopilación es la siguiente:

{
    "date": "fecha del acontecimiento",
    "description": "descripción del evento en cuestión",
    "lang": "lenguaje de la descripción",
    "category1": "catergoría interna del dataset",
    "granularity": "granularidad"
}

Como se puede ver, no cuenta con una estructura compleja, y sus campos más importantes son 'date' que nos indica la fecha del suceso y 'description' donde se encuentran todos los detalles del evento. Este dataset cuenta con 20.330 registros diferentes.

Planteamiento de la problemática y diseño de la solución (tecnologías a implementar)

Se plantea realizar un análisis descriptivo de esta información a nivel de país, agrupando sus eventos históricos y ver qué palabras son recurrentes en estos eventos. Así nos podemos dar una rápida percepción de la historia de un país en concreto. También se plantea analizar palabras clave en los eventos históricos como lo son 'guerra', 'atentados', 'ataque', 'muertos', 'descubrimiento', 'invención' y ver que tan concurrentes son a lo largo de la historia.

Para esta labor, nos apoyaremos de la herramienta MongoDB en su entorno de Python Pymongo. Este sistema de base de datos NoSQL nos ayudará a manejar adecuadamente el formato de este dataset (JSON) y más importante aún con el tratamiento de textos. Para esto último nos apoyaremos en dos funcionalidades de MongoDB: En el uso de expresiones regulares para busqueda en campos de texto y en las operaciones Map-Reduce. Junto con MongoDB, nos apoyaremos en las librerías propias de analítica de datos de Python. Con esto se pretenderá alcanzar los objetivos de este proyecto.

[x]it! support for working with todo and check list files in Sublime Text

[x]it! for Sublime Text This Sublime Package provides syntax-highlighting, shortcuts, and auto-completions for [x]it! files. Features Syntax highlight

Jan Heuermann 18 Sep 19, 2022
Source-o-grapher is a tool built with the aim to investigate software resilience aspects of Open Source Software (OSS) projects.

Source-o-grapher is a tool built with the aim to investigate software resilience aspects of Open Source Software (OSS) projects.

Aristotle University 5 Jun 28, 2022
Script for resizing MTD partitions on a QNAP device in order to be available to upgrade from buster to bullseye

QNAP partitions resize for kirkwood devices. As explained by Marin Michlmayr, Debian bullseye support on kirkwood QNAP devices was dropped due to [mai

Arnaud Mouiche 26 Jan 05, 2023
BlackIP-Rep is a tool designed to gather the reputation and information of Bulk IP's.

BlackIP-Rep is a tool designed to gather the reputation and information of Bulk IP's. Focused on increasing the workflow of Security Operations(SOC) team during investigation.

0LiVEr 6 Dec 12, 2022
CD for MachineLearnia

Codebase supporting my talk on CI/CD for MachineLearnia (Nov 12 2021) The dataset used is available here. The point of the talk is to demonstrate a si

0 Feb 23, 2022
It is Keqin Wang first project in CMU, trying to use DRL(PPO) to control a 5-dof manipulator to draw line in space.

5dof-robot-writing this project aim to use PPO control a 5 dof manipulator to draw lines in 3d space. Introduction to the files the pybullet environme

Keqin Wang 4 Aug 22, 2022
A 100% python file organizer. Keep your computer always organized!

PythonOrganizer A 100% python file organizer. Keep your computer always organized! To run the project, just clone the folder and run the installation

3 Dec 02, 2022
Flask html response minifier

Flask-HTMLmin Minify flask text/html mime type responses. Just add MINIFY_HTML = True to your deployment config to minify HTML and text responses of y

Hamid Feizabadi 85 Dec 07, 2022
Collection of Beginner to Intermediate level Python scripts contributed by members and participants.

Hacktoberfest2021-Python Hello there! This repository contains a 'Collection of Beginner to Intermediate level Python projects', created specially for

12 May 25, 2022
Implements a polyglot REPL which supports multiple languages and shared meta-object protocol scope between REPLs.

MetaCall Polyglot REPL Description This repository implements a Polyglot REPL which shares the state of the meta-object protocol between the REPLs. Us

MetaCall 10 Dec 28, 2022
Create an application to visualize single/multiple Xandar Kardian people counting sensors detection result for a indoor area.

Program Design Purpose: We want to create an application to visualize single/multiple Xandar Kardian people counting sensors detection result for a indoor area.

2 Dec 28, 2022
MODSKIN-LOLPRO-updater: The mod is fkn 10y old and has'nt a self-updater

The mod is fkn 10y old and has'nt a self-updater. To use it just run the exec, wait some seconds, and it will run the new modsk

Shiro Amurha 3 Apr 23, 2022
🍞 Create dynamic spreadsheets with arbitrary layouts using Python

🍞 tartine What this is Installation Usage example Fetching some data Getting started Adding a header Linking more cells Cell formatting API reference

Max Halford 11 Apr 16, 2022
A wide AOI generator tool.

Dark Generator A wide AOI generator tool. Information Installation To Install you have to have python 3.x and pip installed on your system. If you hav

Darkest Surface 12 Dec 26, 2022
Scripts to convert the Ted-MDB corpora into the formats for DISRPT shared task and the converted corpora

Scripts to convert the Ted-MDB corpora into the formats for DISRPT shared task and the converted corpora.

1 Feb 08, 2022
Mixtaper - Web app to make mixtapes

Mixtaper A web app which allows you to input songs in the form of youtube links

suryansh 1 Feb 14, 2022
Winxp_python3.6.15 - Python 3.6.15 For Windows XP SP3

This is Python version 3.6.15 Copyright (c) 2001-2021 Python Software Foundation. All rights reserved. See the end of this file for further copyright

Alex Free 13 Sep 11, 2022
Uma versão em Python/Ursina do aplicativo Real Drum (android).

Real Drum Descrição Esta é uma versão alternativa feita em Python com a engine Ursina do aplicatio Real Drum (presente no Google Play Store). Como exe

hayukimori 5 Aug 20, 2022
Job Guy Backend

جاب‌گای چیست؟ اونجا وضعیت چطوریه؟ یه سوال به همین کلیت و ابهام معمولا وقتی برای یه شرکت رزومه می‌فرستیم این سوال کلی و بزرگ برای همه پیش میاد.اونجا وض

Jobguy.work 217 Dec 25, 2022
Dotfiles for my configurations!

Dotfiles Repo Welcome! Over here, you can find my dotfiles for various applications, including QTile, Alacritty, Kitty, LunarVim, and more! Make sure

class PythonAddict 3 Jan 10, 2022