Avoid PyCharm Overloading
=========================
This speeds up PyCharm's package index processes and avoids CPU & memory
overloading.
On my machine (4c/32GB) with 5 conda environments and 10 projects, this reduces
PyCharm's package scanning from 15 minutes to 45 seconds.
Why?
----
When PyCharm has multiple conda environments, it will sometimes launch concurrent
package scanning across those environments. To do so, it uses the `conda` cli,
and a tool called conda_packaging_tool.py, available from the JetBrains
intellij-community github repository (see below).
Unfortunately, these tools do not behave well when launched in parallel, as
they tend to overload CPU & memory in this case.
How?
----
To avoid stalling these processes, we modify their sources as follows:
1. modify the conda cli
$ conda activate base
$ pip install runfast
$ nano `which conda`
Modify the main section:
if __name__ == '__main__':
from conda.cli import main
from runfast import cached
cached(main)
2. modify the conda_packaging_tool.py
if __name__ == '__main__':
from runfast import cached
cached(main)
How does this work?
-------------------
Two simple steps:
1. When called in parallel, only one process is allowed to proceed.
2. `runfast.cached` caches the output of these tools (stdout, stderr) for 1 minute,
given the same command line parameters
That is, when PyCharm launches 5 package scanning (`conda list`) commands, only
one of them will run immediately. If some of the scans are for the same environment,
only one of them will actually run, while the others simply return the same output.
How to clear the cache or avoid caching at all?
$ export RUNFAST_NOCACHE=1
Testing
-------
a) conda packaging tool
# first time
$ time python /opt/pycharm-community-2021.2.1/plugins/python-ce/helpers/conda_packaging_tool.py
real 0m13.271s
user 0m9.631s
sys 0m1.936s
# second time
$ time python /opt/pycharm-community-2021.2.1/plugins/python-ce/helpers/conda_packaging_tool.py
real 0m3.159s
user 0m0.139s
sys 0m1.298s
b) conda cli
# first time
$ time conda list -p /path/to/env
real 0m7.985s
user 0m7.883s
sys 0m0.087s
# second time
real 0m0.152s
user 0m0.116s
sys 0m0.032s
References
----------
* https://github.com/JetBrains/intellij-community/blob/master/python/helpers/conda_packaging_tool.py
* https://github.com/conda/conda/blob/33a142c16530fcdada6c377486f1c1a385738a96/conda/core/index.py#L53
This speeds up PyCharm's package index processes and avoids CPU & memory overloading
Overview
PyPI package for scaffolding out code for decision tree models that can learn to find relationships between the attributes of an object.
Decision Tree Writer This package allows you to train a binary classification decision tree on a list of labeled dictionaries or class instances, and
My programming language named JoLang. (Mainly created for fun)
JoLang status: not ready So this is my programming language which I decided to name 'JoLang' (inspired by Jonathan and GoLang). Features I implemented
Program Input Nilai Mahasiswa Menggunakan Fungsi
PROGRAM INPUT NILAI MAHASISWA MENGGUNAKAN FUNGSI Nama : Maulana Reza Badrudin Nim : 312110510 Matkul : Bahas Pemograman DESKRIPSI Deklarasi dicti
Craxk is a SINGLE AND NON-REPLICABLE Hash that uses data from the hardware where it is executed to form a hash that can only be reproduced by a single machine.
What is Craxk ? Craxk is a UNIQUE AND NON-REPLICABLE Hash that uses data from the hardware where it is executed to form a hash that can only be reprod
Practice10 - Operasi String With Python
Operasi String MY SOSIAL MEDIA : Apa itu Python String ? String adalah urutan si
A python package that adds "docs" command to disnake
About This extension's purpose is of adding a "docs" command, its purpose is to help documenting in chat. How To Load It from disnake.ext import comma
kurwa deska ADB
kurwa-deska-ADB kurwa-deska Запуск Linux -- python3 kurwa_deska.py Termux -- python3 kurwa_deska.py Встановлення cd kurwa_deska ADB і зразу запуск pyt
→ Plantilla de registro para Python
🔧 Pasos Necesarios CMD 🖥️ SOCKETS pip install sockets 🎨 COLORAMA pip install colorama 💻 Código register-by-inputs from turtle import color # Impor
edgetest is a tox-inspired python library that will loop through your project's dependencies, and check if your project is compatible with the latest version of each dependency
Bleeding edge dependency testing Full Documentation edgetest is a tox-inspired python library that will loop through your project's dependencies, and
Python Osmium Examples
Python Osmium Examples This is a set (currently of size 1) of examples showing practical usage of PyOsmium, a thin wrapper around the osmium library.
TurtleBot Control App - TurtleBot Control App With Python
TURTLEBOT CONTROL APP INDEX: 1. Introduction 2. Environments 2.1. Simulated Envi
A tool for removing PUPs using signatures
Unwanted program removal tool A tool for removing PUPs using signatures What is the unwanted program removal tool? The unwanted program removal tool i
Scripts used in the RayStation medical radiation dosimetry treatment planning system
Med Phys Scripts These are scripts that I, the medical physics assistant at Cookeville Regional Medical Center, wrote for use in our radiation therapy
A replacement of qsreplace, accepts URLs as standard input, replaces all query string values with user-supplied values and stdout.
Bhedak A replacement of qsreplace, accepts URLs as standard input, replaces all query string values with user-supplied values and stdout. Works on eve
Module 2's katas from Launch X's python introduction course.
Module2Katas Module 2's katas from Launch X's python introduction course. Virtual environment creation process (on Windows): Create a folder in any de
MindF**k it's a programming language as BrainFuck, but with some cool features.
MindF**k Description MindF**k it's a programming language as BrainFuck, but with some cool features. Symbol What does symbol mean Next slot Previo
A blazing fast mass certificate generator script for the community ⚡
A simple mass certificate generator script for the community ⚡ Source Code · Docs · Raw Script Docs All you need Certificate Design a simple template
NeurIPS'19: Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting (Pytorch implementation for noisy labels).
Meta-Weight-Net NeurIPS'19: Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting (Official Pytorch implementation for noisy labels). The
Implent of Oracle Base line and Lea-3 Baseline
Oracle-Baseline Implent of Oracle Base line and Lea-3 Baseline Oracle Oracle : This model is used to obtain an oracle with a greedy algorithm similar
Tensorboard plugin 3d with python
tensorboard-plugin-3d Overview In this example, we render a run selector dropdown component. When the user selects a run, it shows a preview of all sc