A utility to search, download and process Landsat 8 satellite imagery

Overview

Landsat-util

https://travis-ci.org/developmentseed/landsat-util.svg?branch=master Downloads License

Landsat-util is a command line utility that makes it easy to search, download, and process Landsat imagery.

Docs

For full documentation visit: https://pythonhosted.org/landsat-util/

To run the documentation locally:

$ pip install -r requirements/dev.txt
$ cd docs
$ make html

Recently Added Features

  • Improved pansharpening
  • Use BQA bands for cloud/snow coverage and use in color correction
  • Add support for different NDVI color maps (three included)
  • Add support for image clipping using the new --clip flag

Change Log

See CHANGES.txt.

Comments
  • Searching returns no result

    Searching returns no result

    Hi people!

    landsat is not returning any results. For example:

    $ landsat search -p 228,81 --latest 5
    {
        "limit": 25, 
        "results": [], 
        "status": "SUCCESS", 
        "total": 0, 
        "total_returned": 0
    }
    ['Search completed!']
    
    $ landsat search -p 23,35 --latest 5
    {
        "limit": 25, 
        "results": [], 
        "status": "SUCCESS", 
        "total": 0, 
        "total_returned": 0
    }
    ['Search completed!']
    
    $ landsat search -p 226,84 --latest 5
    {
        "limit": 25, 
        "results": [], 
        "status": "SUCCESS", 
        "total": 0, 
        "total_returned": 0
    }
    

    Any suggestion/pointer to solve this problem? Thanks in advance!

    Matias

    opened by matiaslee 27
  • doesn't work

    doesn't work

    Its probably a simple fix but I'm not an IT guy. I keep getting the following error

    landsat search --cloud 4 --start "january 1 2014" --end "january 10 2014" -p 009,045 Traceback (most recent call last): File "/Users/schull/anaconda/bin/landsat", line 5, in import landsat.landsat File "/Users/schull/anaconda/lib/python2.7/site-packages/landsat/landsat.py", line 21, in from image import Process, FileDoesNotExist File "/Users/schull/anaconda/lib/python2.7/site-packages/landsat/image.py", line 13, in import rasterio File "/Users/schull/anaconda/lib/python2.7/site-packages/rasterio/init.py", line 8, in from rasterio._base import eval_window, window_shape, window_index ImportError: dlopen(/Users/schull/anaconda/lib/python2.7/site-packages/rasterio/_base.so, 2): Library not loaded: libgdal.1.dylib Referenced from: /Users/schull/anaconda/lib/python2.7/site-packages/rasterio/_base.so Reason: image not found

    Any ideas how to fix it?

    opened by bucricket 19
  • Landsat8 download problem from USGS-EROS for L1C product not available on AWS S3 or Google Storage

    Landsat8 download problem from USGS-EROS for L1C product not available on AWS S3 or Google Storage

    I am not having success trying to load a set of specific Landsat-8 scenes from USGS EROS site -- for scenes not available on AWS or Google-- trying this from an OSX machine.

    I get the scene names using '> landsat search' just fine... With '> landsat download', I get "product not available on AWS S3 or Google Storage"... one of these scenes, as an example, is LC81290582016007LGN02.

    When I supply my USGS EROS credentials --username --password (I do have m2m / special inventory access enabled on this account), I get the error message below. Any help greatly appreciated...

    #---- Time spent : 0.75 seconds Traceback (most recent call last): File "/usr/local/bin/landsat", line 11, in sys.exit(main()) File "/usr/local/lib/python2.7/site-packages/landsat/landsat.py", line 491, in main exit(*main(args)) File "/usr/local/lib/python2.7/site-packages/landsat/landsat.py", line 408, in main files = d.download(args.scenes, bands) File "/usr/local/lib/python2.7/site-packages/landsat/downloader.py", line 80, in download files.append(self.usgs_eros(scene, self.download_dir)) File "/usr/local/lib/python2.7/site-packages/landsat/downloader.py", line 95, in usgs_eros error_tree = ElementTree.fromstring(str(e.message)) File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1311, in XML parser.feed(text) File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1653, in feed self._raiseerror(v) File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1517, in _raiseerror raise err xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 171, column 22

    opened by gybely 12
  • returned non-zero exit status 4

    returned non-zero exit status 4

    Hi. Congratulations on this very usefull tool. I'm having trouble processing the images after dowloading them. Here's the console text:

    "C:\Programs\landsat-util-master>python C:\Programs\landsat-util-master\landsat\l andsat.py search --download --imageprocess --cloud 20 --start "january 1 2014" - -end "april 10 2014" pr 204 032 1 items were found { "limit": 100, "results": [ { "cloud": 4.16, "date": "2014-03-19", "path": "204", "row": "032", "sat_type": "L8", "sceneID": "LC82040322014078LGN00", "thumbnail": "http://earthexplorer.usgs.gov/browse/landsat_8/2014/20 4/032/LC82040322014078LGN00.jpg" } ], "status": "SUCCESS", "total": 1, "total_returned": 1 } Starting the download: Skipping existing item: file://C:\Users\Administrator/landsat/zip/LC820403220140 78LGN00.tar.bz 1 images were downloaded c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00 folder cr eated c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00/warp fold er created c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00/scaled fo lder created c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00/final fol der created Unzipping LC82040322014078LGN00 - It might take some time Creating output file that is 7828P x 7932L. Processing input file c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC8204032 2014078LGN00/LC82040322014078LGN00_B4.TIF. 0...10...20...30...40...50...60...70...80...90...100 - done. Creating output file that is 7828P x 7932L. Processing input file c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC8204032 2014078LGN00/LC82040322014078LGN00_B3.TIF. 0...10...20...30...40...50...60...70...80...90...100 - done. Creating output file that is 7828P x 7932L. Processing input file c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC8204032 2014078LGN00/LC82040322014078LGN00_B2.TIF. 0...10...20...30...40...50...60...70...80...90...100 - done. Creating output file that is 15655P x 15862L. Processing input file c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC8204032 2014078LGN00/LC82040322014078LGN00_B8.TIF. 0...10...20...30...40...50...60...70...80...90...100 - done. Starting the Min/Max process with designated -percent cut- for band 4 of LC82040 322014078LGN00 ... Finished processing band 4 of LC82040322014078LGN00 Starting the Min/Max process with designated -percent cut- for band 3 of LC82040 322014078LGN00 ... Finished processing band 3 of LC82040322014078LGN00 Starting the Min/Max process with designated -percent cut- for band 2 of LC82040 322014078LGN00 ... Finished processing band 2 of LC82040322014078LGN00 scaling pan to min max with 2% cut for band 4 Input file size is 7828, 7932 0...10...20...30...40...50...60...70...80...90...100 - done. scaling pan to min max with 2% cut for band 3 Input file size is 7828, 7932 0...10...20...30...40...50...60...70...80...90...100 - done. scaling pan to min max with 2% cut for band 2 Input file size is 7828, 7932 0...10...20...30...40...50...60...70...80...90...100 - done. Invalid Parameter - -combine Traceback (most recent call last): File "C:\Programs\landsat-util-master\landsat\landsat.py", line 285, in main() File "C:\Programs\landsat-util-master\landsat\landsat.py", line 282, in __main __ main(args) File "C:\Programs\landsat-util-master\landsat\landsat.py", line 236, in main p.full() File "C:\Programs\landsat-util-master\landsat\image_helper.py", line 108, in f ull self._combine() File "C:\Programs\landsat-util-master\landsat\image_helper.py", line 340, in _ combine subprocess.check_call(argv) File "C:\OSGeo4W64\apps\Python27\lib\subprocess.py", line 542, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['convert', '-identify', '-combine', u'c :\users\admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00/war p/LC82040322014078LGN00_B4.TIF', u'c:\users\admini~1\appdata\local\temp\1
    \tmpybepa2/LC82040322014078LGN00/warp/LC82040322014078LGN00_B3.TIF', u'c:\users \admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00/warp/LC8204 0322014078LGN00_B2.TIF', u'c:\users\admini~1\appdata\local\temp\1\tmpybep a2/LC82040322014078LGN00/final/rgb-null.TIF']' returned non-zero exit status 4 "

    Installation Issues v0.2 
    opened by vascobnunes 12
  • landsat collection 1

    landsat collection 1

    Hi,

    I'm wondering if you are planning on implementing the new collection 1 information? The ESPA ordering system no longer stores the pre-collection data and so the sceneID information doesn't work anymore. We need the Landsat-product ID now.

    Mitch

    opened by bucricket 11
  • Loosen requirement versions

    Loosen requirement versions

    I essentially widened the allowed versions for all dependencies, taking care of specifying rasterio<1. The current setup has been tested in all Python >= 3.4 versions (see commit logs). I also updated the Dockerfile, hopefully fixing another issue.

    opened by astrojuanlu 10
  • Support for Copernicus Sentinel-2 imaginery

    Support for Copernicus Sentinel-2 imaginery

    This is just a far fetched open question: would be interesting to support the download / processing of imaginery from the Sentinel-2 satellites of the Copernicus mission? A few points of interest:

    The mission

    Multi-spectral data with 13 bands in the visible, near infrared, and short wave infrared part of the spectrum Systematic global coverage of land surfaces from 56ยฐ S to 84ยฐ N, coastal waters, and all of the Mediterranean Sea Revisiting every 5 days at the Equator under the same viewing conditions Spatial resolution of 10 m, 20 m and 60 m 290 km field of view Free and open data policy

    Imaginery

    Among others, there will be 4 bands at 10m resolution (50% more than Landsat-8): red, green, blue and VNIR. Nicely, there is no need to pansharpen in order to get images at max. resolution.

    bands

    Data access

    Data will be free to access. The good news are that there is an API to perform searches (including geographical bounding, cloud cover, date, etc.) and download quicklooks and products. This API is well documented here. The bad news are that the API requires user registration:

    curl -u username:password "https://scihub.esa.int/dhus/search?q=S2A*&format=json"
    

    Also, the images are distributed in a specific SAFE format which I'm not sure GDAL supports yet (it seems that GDAL supports Sentinel-1 SAFE, but not Sentinel-2 SAFE).


    As you can see the tools are not so ready yet, but I think it could be interesting to have this issue in mind in order to someday extend the functionality of landsat-util, no? The possibility of 50% higher resolution imaginary with 1/3 of the revisit time without the need of pansharpening (which highly increases the resources needed) is very tempting!

    What do you think? =)

    opened by JorgeGT 10
  • update join method for download URL

    update join method for download URL

    replaced join() with "/".join() for building google and Amazon S3 URLs that are compatible across platforms. Remove trailing "/" from S3 and Google base URL in settings.py

    opened by RoboDonut 9
  • Add an option for parallel downloading

    Add an option for parallel downloading

    As I've proposed in Issue #102, these are the modifications in order to allow the use of new command-line flag --use-aria2c true, in order to download the scenes using aria2c's parallel download. This is my first contribution so, in case something looks odd, just let me know, ok? Cheers.

    opened by vbenso 9
  • download band 8 in continuous download/process/pansharpen command

    download band 8 in continuous download/process/pansharpen command

    This command landsat download LC81410412015088LGN00 --bands 432 -p --pansharpen fails because band 8 is not downloaded from aws.

    Have to make sure that band 8 is downloaded if pansharpen flag is used for 2015 and newer imagery.

    opened by scisco 8
  • magick error

    magick error

    After updating Xcode to 4.6.3 and running brew doctor, still i get this error after running the install for landsat-util--has anyone seen this before?

    1 error generated. make[1]: *** [magick/magick_libMagickCore_6_Q16_la-annotate.lo] Error 1 make[1]: *** Waiting for unfinished jobs.... make: *** [install] Error 2

    opened by ebarry 8
  • Coordinates to clip the raster

    Coordinates to clip the raster

    Hi there, I am working on a raster file and want to clip an area from the raster using latitude and longitude. But Landsat API requires coordinates in WGS84 datum. How can I convert these lat and long coordinates into the desired format?. Thanks.

    opened by EhsanAlahi 0
  • Update dependencies for new rasterio version

    Update dependencies for new rasterio version

    Hi,

    Recently rasterio released version 1.0 and is currently at version 1.0.23. That has some important updates and features, and at the moment I cannot install landsat-util because of its rasterio version requirements (currently rasterio~=0.36)

    Would it be possible to update that?

    opened by alessioarena 1
  • Docker search SSL error

    Docker search SSL error

    $ sudo docker run -it -v ~/landsat:/root/landsat developmentseed/landsat-util:latest landsat search --lat 33.9436973 --lon -118.4073388       
    
    /usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
      InsecurePlatformWarning
    Traceback (most recent call last):
      File "/usr/local/bin/landsat", line 9, in <module>
        load_entry_point('landsat-util==0.13.1', 'console_scripts', 'landsat')()
      File "/landsat/landsat/landsat.py", line 488, in __main__
        print(main(args))
      File "/landsat/landsat/landsat.py", line 357, in main
        geojson=args.geojson)
      File "/landsat/landsat/search.py", line 96, in search
        r = requests.get('%s?search=%s&limit=%s' % (self.api_url, search_string, limit))
      File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 69, in get
        return request('get', url, params=params, **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 50, in request
        response = session.request(method=method, url=url, **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 465, in request
        resp = self.send(prep, **send_kwargs)
      File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 573, in send
        r = adapter.send(request, **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 431, in send
        raise SSLError(e, request=request)
    requests.exceptions.SSLError: [Errno 1] _ssl.c:510: error:14077438:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert internal error
    
    opened by palewire 1
  • Problem with using landsat-util downloads lansat8 2018 year data

    Problem with using landsat-util downloads lansat8 2018 year data

    When I tried to download Landsat 8 data with year of 2018 , then there was an error LC81210382018100LGN00 is not available on AWS S3 or Google Storage image

    opened by 120534 0
  • Would like to use landsat-util on windows

    Would like to use landsat-util on windows

    I'm using a windows 10 system. Should I use a docker and run it on Hyper-V? Is this the only way? Anyone familiar with the start-up could help me out? Thanks!

    opened by gracie2017 3
Releases(v0.13.1)
  • v0.13.1(Jan 18, 2017)

  • v0.13.0(Mar 25, 2016)

  • v0.12.2(Mar 25, 2016)

  • v0.12.1(Feb 18, 2016)

  • v0.12.0(Feb 18, 2016)

  • v0.10.0(Jan 5, 2016)

    • add support for bare json output
    • faster travis tests
    • add street search
    • apply ndvigrey to process and download
    • fixes #127
    • update commands help file
    • apply pansharpen and ndvi only if -p is used
    • download zip if bands are not specified
    • better handle url joins to a fix error on Windows
    • other small bug fixes
    Source code(tar.gz)
    Source code(zip)
  • v0.9.1(Jan 5, 2016)

  • v0.8.0(Oct 23, 2015)

    0.8.0 (2015-09-22)

    • Improved docs
    • Add --ndvi flag
    • Handle downloading new bands (10, 11, QA)
    • Improved color correction
    • Remove noise in pansharpened image processing
    Source code(tar.gz)
    Source code(zip)
  • v0.7.0(May 29, 2015)

    • New documentation
    • Deployed to readthedocs
    • Automate deployment to pypi
    • Adds docker support
    • skip unzipping if images already unzipped
    • add force-unzip flag
    • fix a bug where multiple downloads was not followed by multiple process #81
    • fix a bug where if scenes was downloaded from google instead of aws, process failed #84
    • download band 8 when pansharpen fixes #73
    Source code(tar.gz)
    Source code(zip)
  • v0.6.3(Apr 29, 2015)

    includes:

    • adjust lower rescaling bound, closes #66 for now
    • better pixel math for #71
    • update destination transform calculation for reprojection in polar regions
    Source code(tar.gz)
    Source code(zip)
  • v0.6.2(Apr 25, 2015)

  • v0.6.1(Apr 23, 2015)

    Includes:

    • Updated tests that run faster
    • New upload command for uploading final image to S3
    • New command for continuous upload, process and upload
    • Updated image processing that produces better color correction
    • Other bug fixes
    Source code(tar.gz)
    Source code(zip)
  • v0.5.1(Apr 8, 2015)

  • v0.5.0(Mar 10, 2015)

    Version 0.5.0 includes:

    • Fewer dependencies
    • Uses wheel on OSX
    • New image processing algorithm which is faster and require less storage
    • Updated API
    • Includes longitude latitude search
    • Improved console output
    • Accepts bands combinations for processing images
    • Includes a new faster downloader
    • More comprehensive tests (96% coverage)
    • Added to pypi pip install landsat-util
    • Brew formula removed
    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(Aug 28, 2014)

Owner
Development Seed
Development Seed
LicenseLocation - License Location With Python

LicenseLocation Hi,everyone! โค ๐Ÿงก ๐Ÿ’› ๐Ÿ’š ๐Ÿ’™ ๐Ÿ’œ This is my first project! โœ” Actual

The Bin 1 Jan 25, 2022
This repository contains the scripts to derivate the ENU and ECEF coordinates from the longitude, latitude, and altitude values encoded in the NAD83 coordinates.

This repository contains the scripts to derivate the ENU and ECEF coordinates from the longitude, latitude, and altitude values encoded in the NAD83 coordinates.

Luigi Cruz 1 Feb 07, 2022
a Geolocator made in python

Geolocator A Geolocator made in python โœจ Features locates ur location using ur ip thats it! ๐Ÿ’โ€โ™€๏ธ How to use first download the locator.py file instal

Portgas D Ace 1 Oct 27, 2021
Code and coordinates for Matt's 2021 xmas tree

xmastree2021 Code and coordinates for Matt's 2021 xmas tree This repository contains the code and coordinates used for Matt's 2021 Christmas tree, as

Stand-up Maths 117 Jan 01, 2023
A ready-to-use curated list of Spectral Indices for Remote Sensing applications.

A ready-to-use curated list of Spectral Indices for Remote Sensing applications. GitHub: https://github.com/davemlz/awesome-ee-spectral-indices Docume

David Montero Loaiza 488 Jan 03, 2023
Documentation and samples for ArcGIS API for Python

ArcGIS API for Python ArcGIS API for Python is a Python library for working with maps and geospatial data, powered by web GIS. It provides simple and

Esri 1.4k Dec 30, 2022
A NASA MEaSUREs project to provide automated, low latency, global glacier flow and elevation change datasets

Notebooks A NASA MEaSUREs project to provide automated, low latency, global glacier flow and elevation change datasets This repository provides tools

NASA Jet Propulsion Laboratory 27 Oct 25, 2022
Geocoding library for Python.

geopy geopy is a Python client for several popular geocoding web services. geopy makes it easy for Python developers to locate the coordinates of addr

geopy 3.8k Dec 30, 2022
A light-weight, versatile XYZ tile server, built with Flask and Rasterio :earth_africa:

Terracotta is a pure Python tile server that runs as a WSGI app on a dedicated webserver or as a serverless app on AWS Lambda. It is built on a modern

DHI GRAS 531 Dec 28, 2022
๐ŸŒ Local tile server for viewing geospatial raster files with ipyleaflet or folium

๐ŸŒ Local Tile Server for Geospatial Rasters Need to visualize a rather large (gigabytes) raster you have locally? This is for you. A Flask application

Bane Sullivan 192 Jan 04, 2023
PyTorch implementation of ''Background Activation Suppression for Weakly Supervised Object Localization''.

Background Activation Suppression for Weakly Supervised Object Localization PyTorch implementation of ''Background Activation Suppression for Weakly S

34 Dec 27, 2022
Introduction to Geospatial Analysis in Python

Introduction to Geospatial Analysis in Python This repository is in support of a talk on geospatial data. Data To recreate all of the examples, the da

Dillon Gardner 6 Oct 19, 2022
How to use COG's (Cloud optimized GeoTIFFs) with Rasterio

How to use COG's (Cloud optimized GeoTIFFs) with Rasterio According to Cogeo.org: A Cloud Opdtimized GeoTIFF (COG) is a regular GeoTIFF file, aimed at

Marvin Gabler 8 Jul 29, 2022
Color correction plugin for rasterio

rio-color A rasterio plugin for applying basic color-oriented image operations to geospatial rasters. Goals No heavy dependencies: rio-color is purpos

Mapbox 111 Nov 15, 2022
ColoringMapAlgorithm-CSP- - Graphical Coloring of Countries with Condition Satisfaction Algorithm

ColoringMapAlgorithm-CSP- Condition Satisfaction Algorithm Output Condition

Kerem TAN 2 Jan 10, 2022
This GUI app was created to show the detailed information about the weather in any city selected by user

WeatherApp Content Brief description Tools Features Hotkeys How it works Screenshots Ways to improve the project Installation Brief description This G

TheBugYouCantFix 5 Dec 30, 2022
gpdvega is a bridge between GeoPandas and Altair that allows to seamlessly chart geospatial data

gpdvega gpdvega is a bridge between GeoPandas a geospatial extension of Pandas and the declarative statistical visualization library Altair, which all

Ilia Timofeev 49 Jul 25, 2022
Raster processing benchmarks for Python and R packages

Raster processing benchmarks This repository contains a collection of raster processing benchmarks for Python and R packages. The tests cover the most

Krzysztof Dyba 13 Oct 24, 2022
LEOGPS - Satellite Navigation with GPS on Python!

LEOGPS is an open-source Python software which performs relative satellite navigation between two formation flying satellites, with the objective of high accuracy relative positioning. Specifically,

Samuel Low 50 Dec 13, 2022
Example of animated maps in matplotlib + geopandas using entire time series of congressional district maps from UCLA archive. rendered, interactive version below

Example of animated maps in matplotlib + geopandas using entire time series of congressional district maps from UCLA archive. rendered, interactive version below

Apoorva Lal 5 May 18, 2022