The Multi-Mission Maximum Likelihood framework (3ML)

Overview

CI Conda Build and Publish Test Against XSPEC codecov Documentation Status License

GitHub pull requests GitHub issues

PyPi

PyPI version fury.io PyPI - Downloads PyPI - Python Version

Conda

Conda Conda

drawing

The Multi-Mission Maximum Likelihood framework (3ML)

A framework for multi-wavelength/multi-messenger analysis for astronomy/astrophysics.


Astrophysical sources are observed by different instruments at different wavelengths with an unprecedented quality. Putting all these data together to form a coherent view, however, is a very difficult task. Indeed, each instrument and data type has its own ad-hoc software and handling procedure, which present steep learning curves and do not talk to each other.

The Multi-Mission Maximum Likelihood framework (3ML) provides a common high-level interface and model definition, which allows for an easy, coherent and intuitive modeling of sources using all the available data, no matter their origin. At the same time, thanks to its architecture based on plug-ins, 3ML uses under the hood the official software of each instrument, the only one certified and maintained by the collaboration which built the instrument itself. This guarantees that 3ML is always using the best possible methodology to deal with the data of each instrument.

drawing

Though Maximum Likelihood is in the name for historical reasons, 3ML is an interface to several Bayesian inference algorithms such as MCMC and nested sampling as well as likelihood optimization algorithms. Each approach to analysis can be seamlessly switched between allowing users to try different approaches quickly and without having to rewrite their model or data interfaces.

Like your XPSEC models? You can use them in 3ML as well as our growing selection of 1-,2- and 3-D models from our fast and customizable modeling language astromodels.

Installation

Installing with pip or conda is easy

pip install astromodels threeml
conda  install astromodels threeml -c threeml conda-forge 

Please refer to the Installation instructions for more details and trouble-shooting.

Press

Who is using 3ML?

Here is a highlight list of teams and their publications using 3ML.

A full list of publications using 3ML is here.

Citing

If you find this package useful in you analysis, or the code in your own custom data tools, please cite:

Vianello et al. (2015)

Acknowledgements

3ML makes use of the Spanish Virtual Observatory's Filter Profile service (http://svo2.cab.inta-csic.es/svo/theory/fps3/index.php?mode=browse&gname=NIRT).

If you use these profiles in your research, please consider citing them by using the following suggested sentence in your paper:

"This research has made use of the SVO Filter Profile Service (http://svo2.cab.inta-csic.es/theory/fps/) supported from the Spanish MINECO through grant AyA2014-55216"

and citing the following publications:

The SVO Filter Profile Service. Rodrigo, C., Solano, E., Bayo, A. http://ivoa.net/documents/Notes/SVOFPS/index.html The Filter Profile Service Access Protocol. Rodrigo, C., Solano, E. http://ivoa.net/documents/Notes/SVOFPSDAL/index.html

ThreeML is supported by National Science Foundation (NSF)

Comments
  • Use from astromodels Xspec import * error

    Use from astromodels Xspec import * error

    Hello! When I want to use xspec library to combine GBM and XRT to analyze energy spectrum, I try to use from astromodels Xspec import *has the following error messages. How did this happen?


    ImportError Traceback (most recent call last) /tmp/ipykernel_12482/3995023305.py in 5 6 # we will need XPSEC models for extinction ----> 7 from astromodels.xspec import * 8 from astromodels.xspec.xspec_settings import *

    ~/miniconda3/envs/threeML/lib/python3.7/site-packages/astromodels/xspec/init.py in ----> 1 from .factory import * 2 #from .xspec_settings import x

    ~/miniconda3/envs/threeML/lib/python3.7/site-packages/astromodels/xspec/factory.py in 16 pass 17 ---> 18 from astromodels.xspec import _xspec 19 20 # When running in a Anaconda environment, the package xspec-modelsonly

    ImportError: cannot import name '_xspec' from 'astromodels.xspec' (/home/hank/miniconda3/envs/threeML/lib/python3.7/site-packages/astromodels/xspec/init.py)

    opened by ghost 64
  • How do I install a stable threeml module?

    How do I install a stable threeml module?

    Hello,

    The threeml module is currently latest, causing many modules to fail to install, including the Xspexcmodule. So I want to install a more stable threeml module, so how do I do that?

    opened by ghost 18
  • Background Modeling

    Background Modeling

    I've been looking at the background modeling and it may be better to approach it by adding a node onto PointSource:

    ps = PointSource(...,background_shape=Powerlaw() )
    

    Otherwise, we are just adding a function inside of the plugin and have to dig into private members to access the free parameters of that function.

    Is this too involved of an approach @giacomov ?

    opened by grburgess 18
  • properties of the spectra and choosing sources for SpectrumLike

    properties of the spectra and choosing sources for SpectrumLike

    would it make sense to assign some properties in the SpectrumLike plugin. it is quite useful when one wants to associate only some sources to a given spectrum, instead of summing all, as it is done now.

    opened by volodymyrss 17
  • XS_gabs model is not correctly imported

    XS_gabs model is not correctly imported

    I am not sure yet what is the cause, but the model takes only 0 or 1 values, no intermediate, independently of the strength. Position and width still work, but fit breaks, naturally, due to lack of sensitivity to strength. cyclabs e.g. works well (but I don't need it now).

    opened by volodymyrss 17
  • Installation with Fermitools

    Installation with Fermitools

    Discussed in https://github.com/threeML/threeML/discussions/576

    Originally posted by PepeJoseHU September 11, 2022 Hello, I plan to install 3ML for Fermi data with CONDA, but looks 3ML have dependence with fermitools. But when I install 'fermitools', it also make an environment with CONDA, so how to fix this problem or if there have way to make connection between different CONDA environments?

    Thanks!

    opened by grburgess 16
  • Response and GBM files

    Response and GBM files

    There seems to be an issue with the reading of GBM DRMs (similar to what I have seen with gammapy)

    
    /usr/local/Cellar/python/HEAD/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/threeML-0.3.2-py2.7.egg/threeML/plugins/ogip.pyc in _read_matrix(self, data, n_channels)
        750             for j in range(n_grp[i]):
        751 
    --> 752                 rsp[i, f_chan[i][j]: f_chan[i][j] + n_chan[i][j]] = matrix[i][m_start:m_start + n_chan[i][j]]
        753 
        754                 m_start += n_chan[i][j]
    
    ValueError: could not broadcast input array from shape (128) into shape (127)
    
    
    
    bug 
    opened by grburgess 16
  • Plotting

    Plotting

    I propose some updates to plotting routines...

    1. Added a function to joint_likelihood.py that will return 1D/2D likelihood profiles/contours for all parameters/combinations of parameters for convenience.

    2. Added an option to plot_point_source_spectra() to also plot extended source spectra (this will just plot whatever was set as the spectral shape, i.e. the 'spatially integrated' spectrum.

    3a. Moved convergence_plots() and plot_chains() from bayesian_analysis.py to analysis_results.py, so that the plots can be produced from analysis results loaded from disk. 3b. Changed convergence_plots() to use the same number of samples for the bootstrapping average as for the sliding window, I hope that was the original intent.

    opened by henrikef 15
  • added upper/lower limits

    added upper/lower limits

    Added three functions based on the equal_tail_intervals to get the X percent upper/lower limits.

    This makes things rather convenient for finding lower limits on cutoffs (or flux normalizations) after doing Bayesian analysis, although I suppose it can also be used in a frequentist analysis once the variates are computed.

    opened by cbrisboi 13
  • "Current minimum stored after fit and current do not correspond!" messages with all PyGMO algorithms

    I am getting "Current minimum stored after fit and current do not correspond!" messages with all PyGMO algorithms for a very simple model and data, which works with MINUIT.

    opened by volodymyrss 12
  • Alternative to root_numpy

    Alternative to root_numpy

    I was looking at updating hawc_hal to use uproot rather than the old ROOT interface for its .root file io and saw that a big part of the root_numpy dependency is carried over from 3ML via the dependency on threeML/io/cern_root_utils. I would generally prefer to stay consistent with file handling between 3ML and hawc_hal. Is there a preference for uproot vs the new ROOT interface? This is already an issue in hawc_hal, see https://github.com/threeML/hawc_hal/issues/42

    The code looks straightforward to update, but before I run off and implement something that is not preferred, I thought asking would be wise first.

    no-issue-activity 
    opened by cbrisboi 11
  • "FitFailed: MIGRAD call failed. This is usually due to unconstrained parameters."

    Hi,

    I'm running into the following fitting error while trying to fit just Fermi-LAT data: "FitFailed: MIGRAD call failed. This is usually due to unconstrained parameters."

    I'm primarily interested in using 3ML to create joint fits of Fermi-LAT data with X-ray (NuSTAR/NICER/XMM/etc) data. I have already downloaded and created intermediate (ltcube, expo maps, etc). To recreate my error: I create a data from the DataList like so:

    eventFile = os.path.join( 'photon_file.fits' ) ft2File = os.path.join('spacecraft_file.fits' ) expomap = os.path.join( 'bexpmap_00.fits' ) ltcube = os.path.join( 'ltcube_00.fits' )

    if is_plugin_available("FermiLATLike"):

    LAT = FermiLATLike( "LAT", eventFile, ft2File, ltcube, 'binned', binned_expo_map=expomap, source_maps=srcmap )
    #I'm performing binned analysis on my source and have created the binned exposure and src maps.
    

    data = DataList(LAT)

    #%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%

    Now I define my spectral model:

    spectrum=Log_parabola() source1=PointSource(test_src,ra=ra,dec=dec,spectral_shape=spectrum)

    model=Model(source1)

    fluxUnit=1/(u.MeVu.cm**2u.s)

    model.test_src.spectrum.main.Log_parabola.alpha.value=-2.7 model.test_src.spectrum.main.Log_parabola.alpha.bounds=(-5.0,-0.5) model.test_src.spectrum.main.Log_parabola.alpha.fix= False

    model.test_src.spectrum.main.Log_parabola.piv.value= 2134*u.MeV model.test_src.spectrum.main.Log_parabola.piv.fix= True

    model.test_src.spectrum.main.Log_parabola.K.value=2.2e-6fluxUnit model.test_src.spectrum.main.Log_parabola.K.bounds=(1e-12fluxUnit,1e1*fluxUnit) model.test_src.spectrum.main.Log_parabola.K.fix= False

    model.test_src.spectrum.main.Log_parabola.beta.value=0.3 model.test_src.spectrum.main.Log_parabola.beta.bounds=(0.0,1.0) model.test_src.spectrum.main.Log_parabola.beta.fix= False

    #%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%

    And start the likelihood analysis: jl = JointLikelihood(model, data) res = jl.fit()

    #%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%

    At this point, I run into the aforementioned error: "FitFailed: MIGRAD call failed. This is usually due to unconstrained parameters."

    When investigating the model, there don't seem to be any constraints on the normalization (but this may be a product of assigning it a unit?):

                                           value min_value max_value  \
    

    test_src.spectrum.main.Log_parabola.K 0.0 0.0 0.0

    Any help would be appreciated.

    opened by aplangrangian 0
  • Graphics bug - Kills kernel

    Graphics bug - Kills kernel

    Describe the bug Importing TriggerSelector from GtBurst AFTER importing threeML causes crashes when viewing lightcurves or other plots. Specifically, after importing threeML then TriggerSelector, if one downloads GBM data, creates a TimeSeriesBuilder, and then goes to view the lightcurve, a Python3.9 window pops up but is unresponsive, and after force quitting it, the kernel also dies.

    NOTE: This is for BUG reports and not questions about generic data analysis.

    To Reproduce

    Run these commands in jupyter notebook in sequential order:

    from threeML import *

    from GtBurst.TriggerSelector import TriggerSelector

    gbm_data = download_GBM_trigger_data("bn090926181",detectors=['n3'])

    ts1 = TimeSeriesBuilder.from_gbm_cspec_or_ctime('n3', cspec_or_ctime_file=gbm_data['n3']["cspec"], rsp_file=gbm_data['n3']["rsp"] )

    fig = ts1.view_lightcurve(-50,100);

    Expected behavior I expect to view a light curve in the notebook, but a python3.9 unresponsive window pops up, and the kernel dies. This is fixed when importing TriggerSelector first, and then importing threeML.

    Screenshots Screen Shot 2022-12-16 at 11 11 28 AM

    Desktop (please complete the following information):

    • OS: iOS
    • Montery version 12.6
    • Apple M1 Pro chip
    opened by nkirschner 1
  • Finishing CastorLike and Ability to Build flux points.

    Finishing CastorLike and Ability to Build flux points.

    Feature Request

    I have been working on cleaning up and speeding up the castorLike plugin which is designed to fit flux points that have been generated by likelihood profiles (ew). However, I never use this feature, so it would be good to have some help.

    Describe the solution you'd like

    1. Clean up plotting
    2. Documentation and examples of usage
    3. We should build a tool that can actually generate these profiles from any spectral fit. Is it possible from just the FitResults alone?

    @henrikef @giacomov @omodei

    You peeps are probably the experts here. @giacomov started it off, so he might have some directions. Perhaps Eric Charles could be helpful as well?

    help wanted plugin issue 
    opened by grburgess 0
  • reuse extended event class files too

    reuse extended event class files too

    I added a line to include the extended LAT files with a different name also to the list of files to be checked if they were previously downloaded already.

    opened by maklinger 1
  • asymmetric error in XY like

    asymmetric error in XY like

    I have spectral points which have asymmetric error and I am hoping to use XY Like to fit those spectral points. However, it seems like XY Like only supports symmetric errors. Is it possible to make XY Like support asymmetric error?

    opened by jasonfan1997 15
  • Value of BIC

    Value of BIC

    Discussed in https://github.com/threeML/threeML/discussions/585

    Originally posted by changxuezhao September 28, 2022 Dear Dr. Burgess, I have two questions for you about 3ML. First of all, the statistical (whether chi2 or pgstat or BIC) obtained using XSPEC will be smaller than 3ML, I think this is the reason for the likelihood, my question is why the likelihood of 3ML is larger than other tools(xspec,rmfit). Second, why can only the line graph of vFv be drawn without data points, and how to add data points to the vFv graph. Thanks and looking forward to your reply。

    no-issue-activity statistics 
    opened by grburgess 1
Releases(v0.1.0)
  • v0.1.0(Apr 19, 2016)

  • v0.0.5(Feb 6, 2015)

    All dependencies are now explicitly stated in the setup.py file, with the exception of ROOT (we might get rid of this dependency and get MINUIT from somewhere else)

    Source code(tar.gz)
    Source code(zip)
  • v0.0.4(Feb 6, 2015)

    The user can install this version by using pip ("pip install threeML").

    The dependencies (numpy, scipy, numexpr, numdifftools...) must be installed separately.

    Source code(tar.gz)
    Source code(zip)
Owner
The Multi-Mission Maximum Likelihood (3ML)
The Multi-Mission Maximum Likelihood (3ML)
The Curious Layperson: Fine-Grained Image Recognition without Expert Labels (BMVC 2021)

The Curious Layperson: Fine-Grained Image Recognition without Expert Labels Subhabrata Choudhury, Iro Laina, Christian Rupprecht, Andrea Vedaldi Code

Subhabrata Choudhury 18 Dec 27, 2022
Deep Learning for 3D Point Clouds: A Survey (IEEE TPAMI, 2020)

🔥Deep Learning for 3D Point Clouds (IEEE TPAMI, 2020)

Qingyong 1.4k Jan 08, 2023
Some code of the implements of Geological Modeling Using 3D Pixel-Adaptive and Deformable Convolutional Neural Network

3D-GMPDCNN Geological Modeling Using 3D Pixel-Adaptive and Deformable Convolutional Neural Network PyTorch implementation of "Geological Modeling Usin

5 Nov 21, 2022
Face Recognition Attendance Project

Face-Recognition-Attendance-Project In This Project You will learn how to mark attendance using face recognition, Hello Guys This is Gautam Kumar, Thi

Gautam Kumar 1 Dec 03, 2022
Jingju baseline - A baseline model of our project of Beijing opera script generation

Jingju Baseline It is a baseline of our project about Beijing opera script gener

midon 1 Jan 14, 2022
The official MegEngine implementation of the ICCV 2021 paper: GyroFlow: Gyroscope-Guided Unsupervised Optical Flow Learning

[ICCV 2021] GyroFlow: Gyroscope-Guided Unsupervised Optical Flow Learning This is the official implementation of our ICCV2021 paper GyroFlow. Our pres

MEGVII Research 36 Sep 07, 2022
A Simplied Framework of GAN Inversion

Framework of GAN Inversion Introcuction You can implement your own inversion idea using our repo. We offer a full range of tuning settings (in hparams

Kangneng Zhou 13 Sep 27, 2022
LTR_CrossEncoder: Legal Text Retrieval Zalo AI Challenge 2021

LTR_CrossEncoder: Legal Text Retrieval Zalo AI Challenge 2021 We propose a cross encoder model (LTR_CrossEncoder) for information retrieval, re-retrie

Hieu Duong 7 Jan 12, 2022
The code for paper "Learning Implicit Fields for Generative Shape Modeling".

implicit-decoder The tensorflow code for paper "Learning Implicit Fields for Generative Shape Modeling", Zhiqin Chen, Hao (Richard) Zhang. Project pag

Zhiqin Chen 353 Dec 30, 2022
Official Implementation of DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation

DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation [Arxiv] [Paper] As acquiring pixel-wise an

Lukas Hoyer 305 Dec 29, 2022
Improving Transferability of Representations via Augmentation-Aware Self-Supervision

Improving Transferability of Representations via Augmentation-Aware Self-Supervision Accepted to NeurIPS 2021 TL;DR: Learning augmentation-aware infor

hankook 38 Sep 16, 2022
PatrickStar enables Larger, Faster, Greener Pretrained Models for NLP. Democratize AI for everyone.

PatrickStar: Parallel Training of Large Language Models via a Chunk-based Memory Management Meeting PatrickStar Pre-Trained Models (PTM) are becoming

Tencent 633 Dec 28, 2022
Pytorch implementations of the paper Value Functions Factorization with Latent State Information Sharing in Decentralized Multi-Agent Policy Gradients

LSF-SAC Pytorch implementations of the paper Value Functions Factorization with Latent State Information Sharing in Decentralized Multi-Agent Policy G

Hanhan 2 Aug 14, 2022
ktrain is a Python library that makes deep learning and AI more accessible and easier to apply

Overview | Tutorials | Examples | Installation | FAQ | How to Cite Welcome to ktrain News and Announcements 2020-11-08: ktrain v0.25.x is released and

Arun S. Maiya 1.1k Jan 02, 2023
Python port of R's Comprehensive Dynamic Time Warp algorithm package

Welcome to the dtw-python package Comprehensive implementation of Dynamic Time Warping algorithms. DTW is a family of algorithms which compute the loc

Dynamic Time Warping algorithms 154 Dec 26, 2022
DSTC10 Track 2 - Knowledge-grounded Task-oriented Dialogue Modeling on Spoken Conversations

DSTC10 Track 2 - Knowledge-grounded Task-oriented Dialogue Modeling on Spoken Conversations This repository contains the data, scripts and baseline co

Alexa 51 Dec 17, 2022
Repo for FUZE project. I will also publish some Linux kernel LPE exploits for various real world kernel vulnerabilities here. the samples are uploaded for education purposes for red and blue teams.

Linux_kernel_exploits Some Linux kernel exploits for various real world kernel vulnerabilities here. More exploits are yet to come. This repo contains

Wei Wu 472 Dec 21, 2022
Neural Nano-Optics for High-quality Thin Lens Imaging

Neural Nano-Optics for High-quality Thin Lens Imaging Project Page | Paper | Data Ethan Tseng, Shane Colburn, James Whitehead, Luocheng Huang, Seung-H

Ethan Tseng 39 Dec 05, 2022
Steerable discovery of neural audio effects

Steerable discovery of neural audio effects Christian J. Steinmetz and Joshua D. Reiss Abstract Applications of deep learning for audio effects often

Christian J. Steinmetz 182 Dec 29, 2022
CM-NAS: Cross-Modality Neural Architecture Search for Visible-Infrared Person Re-Identification (ICCV2021)

CM-NAS Official Pytorch code of paper CM-NAS: Cross-Modality Neural Architecture Search for Visible-Infrared Person Re-Identification in ICCV2021. Vis

JDAI-CV 40 Nov 25, 2022