A utility to search, download and process Landsat 8 satellite imagery

Overview

Landsat-util

https://travis-ci.org/developmentseed/landsat-util.svg?branch=master Downloads License

Landsat-util is a command line utility that makes it easy to search, download, and process Landsat imagery.

Docs

For full documentation visit: https://pythonhosted.org/landsat-util/

To run the documentation locally:

$ pip install -r requirements/dev.txt
$ cd docs
$ make html

Recently Added Features

  • Improved pansharpening
  • Use BQA bands for cloud/snow coverage and use in color correction
  • Add support for different NDVI color maps (three included)
  • Add support for image clipping using the new --clip flag

Change Log

See CHANGES.txt.

Comments
  • Searching returns no result

    Searching returns no result

    Hi people!

    landsat is not returning any results. For example:

    $ landsat search -p 228,81 --latest 5
    {
        "limit": 25, 
        "results": [], 
        "status": "SUCCESS", 
        "total": 0, 
        "total_returned": 0
    }
    ['Search completed!']
    
    $ landsat search -p 23,35 --latest 5
    {
        "limit": 25, 
        "results": [], 
        "status": "SUCCESS", 
        "total": 0, 
        "total_returned": 0
    }
    ['Search completed!']
    
    $ landsat search -p 226,84 --latest 5
    {
        "limit": 25, 
        "results": [], 
        "status": "SUCCESS", 
        "total": 0, 
        "total_returned": 0
    }
    

    Any suggestion/pointer to solve this problem? Thanks in advance!

    Matias

    opened by matiaslee 27
  • doesn't work

    doesn't work

    Its probably a simple fix but I'm not an IT guy. I keep getting the following error

    landsat search --cloud 4 --start "january 1 2014" --end "january 10 2014" -p 009,045 Traceback (most recent call last): File "/Users/schull/anaconda/bin/landsat", line 5, in import landsat.landsat File "/Users/schull/anaconda/lib/python2.7/site-packages/landsat/landsat.py", line 21, in from image import Process, FileDoesNotExist File "/Users/schull/anaconda/lib/python2.7/site-packages/landsat/image.py", line 13, in import rasterio File "/Users/schull/anaconda/lib/python2.7/site-packages/rasterio/init.py", line 8, in from rasterio._base import eval_window, window_shape, window_index ImportError: dlopen(/Users/schull/anaconda/lib/python2.7/site-packages/rasterio/_base.so, 2): Library not loaded: libgdal.1.dylib Referenced from: /Users/schull/anaconda/lib/python2.7/site-packages/rasterio/_base.so Reason: image not found

    Any ideas how to fix it?

    opened by bucricket 19
  • Landsat8 download problem from USGS-EROS for L1C product not available on AWS S3 or Google Storage

    Landsat8 download problem from USGS-EROS for L1C product not available on AWS S3 or Google Storage

    I am not having success trying to load a set of specific Landsat-8 scenes from USGS EROS site -- for scenes not available on AWS or Google-- trying this from an OSX machine.

    I get the scene names using '> landsat search' just fine... With '> landsat download', I get "product not available on AWS S3 or Google Storage"... one of these scenes, as an example, is LC81290582016007LGN02.

    When I supply my USGS EROS credentials --username --password (I do have m2m / special inventory access enabled on this account), I get the error message below. Any help greatly appreciated...

    #---- Time spent : 0.75 seconds Traceback (most recent call last): File "/usr/local/bin/landsat", line 11, in sys.exit(main()) File "/usr/local/lib/python2.7/site-packages/landsat/landsat.py", line 491, in main exit(*main(args)) File "/usr/local/lib/python2.7/site-packages/landsat/landsat.py", line 408, in main files = d.download(args.scenes, bands) File "/usr/local/lib/python2.7/site-packages/landsat/downloader.py", line 80, in download files.append(self.usgs_eros(scene, self.download_dir)) File "/usr/local/lib/python2.7/site-packages/landsat/downloader.py", line 95, in usgs_eros error_tree = ElementTree.fromstring(str(e.message)) File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1311, in XML parser.feed(text) File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1653, in feed self._raiseerror(v) File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1517, in _raiseerror raise err xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 171, column 22

    opened by gybely 12
  • returned non-zero exit status 4

    returned non-zero exit status 4

    Hi. Congratulations on this very usefull tool. I'm having trouble processing the images after dowloading them. Here's the console text:

    "C:\Programs\landsat-util-master>python C:\Programs\landsat-util-master\landsat\l andsat.py search --download --imageprocess --cloud 20 --start "january 1 2014" - -end "april 10 2014" pr 204 032 1 items were found { "limit": 100, "results": [ { "cloud": 4.16, "date": "2014-03-19", "path": "204", "row": "032", "sat_type": "L8", "sceneID": "LC82040322014078LGN00", "thumbnail": "http://earthexplorer.usgs.gov/browse/landsat_8/2014/20 4/032/LC82040322014078LGN00.jpg" } ], "status": "SUCCESS", "total": 1, "total_returned": 1 } Starting the download: Skipping existing item: file://C:\Users\Administrator/landsat/zip/LC820403220140 78LGN00.tar.bz 1 images were downloaded c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00 folder cr eated c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00/warp fold er created c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00/scaled fo lder created c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00/final fol der created Unzipping LC82040322014078LGN00 - It might take some time Creating output file that is 7828P x 7932L. Processing input file c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC8204032 2014078LGN00/LC82040322014078LGN00_B4.TIF. 0...10...20...30...40...50...60...70...80...90...100 - done. Creating output file that is 7828P x 7932L. Processing input file c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC8204032 2014078LGN00/LC82040322014078LGN00_B3.TIF. 0...10...20...30...40...50...60...70...80...90...100 - done. Creating output file that is 7828P x 7932L. Processing input file c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC8204032 2014078LGN00/LC82040322014078LGN00_B2.TIF. 0...10...20...30...40...50...60...70...80...90...100 - done. Creating output file that is 15655P x 15862L. Processing input file c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC8204032 2014078LGN00/LC82040322014078LGN00_B8.TIF. 0...10...20...30...40...50...60...70...80...90...100 - done. Starting the Min/Max process with designated -percent cut- for band 4 of LC82040 322014078LGN00 ... Finished processing band 4 of LC82040322014078LGN00 Starting the Min/Max process with designated -percent cut- for band 3 of LC82040 322014078LGN00 ... Finished processing band 3 of LC82040322014078LGN00 Starting the Min/Max process with designated -percent cut- for band 2 of LC82040 322014078LGN00 ... Finished processing band 2 of LC82040322014078LGN00 scaling pan to min max with 2% cut for band 4 Input file size is 7828, 7932 0...10...20...30...40...50...60...70...80...90...100 - done. scaling pan to min max with 2% cut for band 3 Input file size is 7828, 7932 0...10...20...30...40...50...60...70...80...90...100 - done. scaling pan to min max with 2% cut for band 2 Input file size is 7828, 7932 0...10...20...30...40...50...60...70...80...90...100 - done. Invalid Parameter - -combine Traceback (most recent call last): File "C:\Programs\landsat-util-master\landsat\landsat.py", line 285, in main() File "C:\Programs\landsat-util-master\landsat\landsat.py", line 282, in __main __ main(args) File "C:\Programs\landsat-util-master\landsat\landsat.py", line 236, in main p.full() File "C:\Programs\landsat-util-master\landsat\image_helper.py", line 108, in f ull self._combine() File "C:\Programs\landsat-util-master\landsat\image_helper.py", line 340, in _ combine subprocess.check_call(argv) File "C:\OSGeo4W64\apps\Python27\lib\subprocess.py", line 542, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['convert', '-identify', '-combine', u'c :\users\admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00/war p/LC82040322014078LGN00_B4.TIF', u'c:\users\admini~1\appdata\local\temp\1
    \tmpybepa2/LC82040322014078LGN00/warp/LC82040322014078LGN00_B3.TIF', u'c:\users \admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00/warp/LC8204 0322014078LGN00_B2.TIF', u'c:\users\admini~1\appdata\local\temp\1\tmpybep a2/LC82040322014078LGN00/final/rgb-null.TIF']' returned non-zero exit status 4 "

    Installation Issues v0.2 
    opened by vascobnunes 12
  • landsat collection 1

    landsat collection 1

    Hi,

    I'm wondering if you are planning on implementing the new collection 1 information? The ESPA ordering system no longer stores the pre-collection data and so the sceneID information doesn't work anymore. We need the Landsat-product ID now.

    Mitch

    opened by bucricket 11
  • Loosen requirement versions

    Loosen requirement versions

    I essentially widened the allowed versions for all dependencies, taking care of specifying rasterio<1. The current setup has been tested in all Python >= 3.4 versions (see commit logs). I also updated the Dockerfile, hopefully fixing another issue.

    opened by astrojuanlu 10
  • Support for Copernicus Sentinel-2 imaginery

    Support for Copernicus Sentinel-2 imaginery

    This is just a far fetched open question: would be interesting to support the download / processing of imaginery from the Sentinel-2 satellites of the Copernicus mission? A few points of interest:

    The mission

    Multi-spectral data with 13 bands in the visible, near infrared, and short wave infrared part of the spectrum Systematic global coverage of land surfaces from 56° S to 84° N, coastal waters, and all of the Mediterranean Sea Revisiting every 5 days at the Equator under the same viewing conditions Spatial resolution of 10 m, 20 m and 60 m 290 km field of view Free and open data policy

    Imaginery

    Among others, there will be 4 bands at 10m resolution (50% more than Landsat-8): red, green, blue and VNIR. Nicely, there is no need to pansharpen in order to get images at max. resolution.

    bands

    Data access

    Data will be free to access. The good news are that there is an API to perform searches (including geographical bounding, cloud cover, date, etc.) and download quicklooks and products. This API is well documented here. The bad news are that the API requires user registration:

    curl -u username:password "https://scihub.esa.int/dhus/search?q=S2A*&format=json"
    

    Also, the images are distributed in a specific SAFE format which I'm not sure GDAL supports yet (it seems that GDAL supports Sentinel-1 SAFE, but not Sentinel-2 SAFE).


    As you can see the tools are not so ready yet, but I think it could be interesting to have this issue in mind in order to someday extend the functionality of landsat-util, no? The possibility of 50% higher resolution imaginary with 1/3 of the revisit time without the need of pansharpening (which highly increases the resources needed) is very tempting!

    What do you think? =)

    opened by JorgeGT 10
  • update join method for download URL

    update join method for download URL

    replaced join() with "/".join() for building google and Amazon S3 URLs that are compatible across platforms. Remove trailing "/" from S3 and Google base URL in settings.py

    opened by RoboDonut 9
  • Add an option for parallel downloading

    Add an option for parallel downloading

    As I've proposed in Issue #102, these are the modifications in order to allow the use of new command-line flag --use-aria2c true, in order to download the scenes using aria2c's parallel download. This is my first contribution so, in case something looks odd, just let me know, ok? Cheers.

    opened by vbenso 9
  • download band 8 in continuous download/process/pansharpen command

    download band 8 in continuous download/process/pansharpen command

    This command landsat download LC81410412015088LGN00 --bands 432 -p --pansharpen fails because band 8 is not downloaded from aws.

    Have to make sure that band 8 is downloaded if pansharpen flag is used for 2015 and newer imagery.

    opened by scisco 8
  • magick error

    magick error

    After updating Xcode to 4.6.3 and running brew doctor, still i get this error after running the install for landsat-util--has anyone seen this before?

    1 error generated. make[1]: *** [magick/magick_libMagickCore_6_Q16_la-annotate.lo] Error 1 make[1]: *** Waiting for unfinished jobs.... make: *** [install] Error 2

    opened by ebarry 8
  • Coordinates to clip the raster

    Coordinates to clip the raster

    Hi there, I am working on a raster file and want to clip an area from the raster using latitude and longitude. But Landsat API requires coordinates in WGS84 datum. How can I convert these lat and long coordinates into the desired format?. Thanks.

    opened by EhsanAlahi 0
  • Update dependencies for new rasterio version

    Update dependencies for new rasterio version

    Hi,

    Recently rasterio released version 1.0 and is currently at version 1.0.23. That has some important updates and features, and at the moment I cannot install landsat-util because of its rasterio version requirements (currently rasterio~=0.36)

    Would it be possible to update that?

    opened by alessioarena 1
  • Docker search SSL error

    Docker search SSL error

    $ sudo docker run -it -v ~/landsat:/root/landsat developmentseed/landsat-util:latest landsat search --lat 33.9436973 --lon -118.4073388       
    
    /usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
      InsecurePlatformWarning
    Traceback (most recent call last):
      File "/usr/local/bin/landsat", line 9, in <module>
        load_entry_point('landsat-util==0.13.1', 'console_scripts', 'landsat')()
      File "/landsat/landsat/landsat.py", line 488, in __main__
        print(main(args))
      File "/landsat/landsat/landsat.py", line 357, in main
        geojson=args.geojson)
      File "/landsat/landsat/search.py", line 96, in search
        r = requests.get('%s?search=%s&limit=%s' % (self.api_url, search_string, limit))
      File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 69, in get
        return request('get', url, params=params, **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 50, in request
        response = session.request(method=method, url=url, **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 465, in request
        resp = self.send(prep, **send_kwargs)
      File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 573, in send
        r = adapter.send(request, **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 431, in send
        raise SSLError(e, request=request)
    requests.exceptions.SSLError: [Errno 1] _ssl.c:510: error:14077438:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert internal error
    
    opened by palewire 1
  • Problem with using landsat-util downloads lansat8 2018 year data

    Problem with using landsat-util downloads lansat8 2018 year data

    When I tried to download Landsat 8 data with year of 2018 , then there was an error LC81210382018100LGN00 is not available on AWS S3 or Google Storage image

    opened by 120534 0
  • Would like to use landsat-util on windows

    Would like to use landsat-util on windows

    I'm using a windows 10 system. Should I use a docker and run it on Hyper-V? Is this the only way? Anyone familiar with the start-up could help me out? Thanks!

    opened by gracie2017 3
Releases(v0.13.1)
  • v0.13.1(Jan 18, 2017)

  • v0.13.0(Mar 25, 2016)

  • v0.12.2(Mar 25, 2016)

  • v0.12.1(Feb 18, 2016)

  • v0.12.0(Feb 18, 2016)

  • v0.10.0(Jan 5, 2016)

    • add support for bare json output
    • faster travis tests
    • add street search
    • apply ndvigrey to process and download
    • fixes #127
    • update commands help file
    • apply pansharpen and ndvi only if -p is used
    • download zip if bands are not specified
    • better handle url joins to a fix error on Windows
    • other small bug fixes
    Source code(tar.gz)
    Source code(zip)
  • v0.9.1(Jan 5, 2016)

  • v0.8.0(Oct 23, 2015)

    0.8.0 (2015-09-22)

    • Improved docs
    • Add --ndvi flag
    • Handle downloading new bands (10, 11, QA)
    • Improved color correction
    • Remove noise in pansharpened image processing
    Source code(tar.gz)
    Source code(zip)
  • v0.7.0(May 29, 2015)

    • New documentation
    • Deployed to readthedocs
    • Automate deployment to pypi
    • Adds docker support
    • skip unzipping if images already unzipped
    • add force-unzip flag
    • fix a bug where multiple downloads was not followed by multiple process #81
    • fix a bug where if scenes was downloaded from google instead of aws, process failed #84
    • download band 8 when pansharpen fixes #73
    Source code(tar.gz)
    Source code(zip)
  • v0.6.3(Apr 29, 2015)

    includes:

    • adjust lower rescaling bound, closes #66 for now
    • better pixel math for #71
    • update destination transform calculation for reprojection in polar regions
    Source code(tar.gz)
    Source code(zip)
  • v0.6.2(Apr 25, 2015)

  • v0.6.1(Apr 23, 2015)

    Includes:

    • Updated tests that run faster
    • New upload command for uploading final image to S3
    • New command for continuous upload, process and upload
    • Updated image processing that produces better color correction
    • Other bug fixes
    Source code(tar.gz)
    Source code(zip)
  • v0.5.1(Apr 8, 2015)

  • v0.5.0(Mar 10, 2015)

    Version 0.5.0 includes:

    • Fewer dependencies
    • Uses wheel on OSX
    • New image processing algorithm which is faster and require less storage
    • Updated API
    • Includes longitude latitude search
    • Improved console output
    • Accepts bands combinations for processing images
    • Includes a new faster downloader
    • More comprehensive tests (96% coverage)
    • Added to pypi pip install landsat-util
    • Brew formula removed
    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(Aug 28, 2014)

Owner
Development Seed
Development Seed
Python library to visualize circular plasmid maps

Plasmidviewer Plasmidviewer is a Python library to visualize plasmid maps from GenBank. This library provides only the function to visualize circular

Mori Hideto 9 Dec 04, 2022
Stitch image tiles into larger composite TIFs

untiler Utility to take a directory of {z}/{x}/{y}.(jpg|png) tiles, and stitch into a scenetiff (tif w/ exact merc tile bounds). Future versions will

Mapbox 38 Dec 16, 2022
Code and coordinates for Matt's 2021 xmas tree

xmastree2021 Code and coordinates for Matt's 2021 xmas tree This repository contains the code and coordinates used for Matt's 2021 Christmas tree, as

Stand-up Maths 117 Jan 01, 2023
geobeam - adds GIS capabilities to your Apache Beam and Dataflow pipelines.

geobeam adds GIS capabilities to your Apache Beam pipelines. What does geobeam do? geobeam enables you to ingest and analyze massive amounts of geospa

Google Cloud Platform 61 Nov 08, 2022
Advanced raster and geometry manipulations

buzzard In a nutshell, the buzzard library provides powerful abstractions to manipulate together images and geometries that come from different kind o

Earthcube Lab 30 Jun 20, 2022
Tool to display your current position and angle above your radar

🛠 Tool to display your current position and angle above your radar. As a response to the CS:GO Update on 1.2.2022, which makes cl_showpos a cheat-pro

Miko 6 Jan 04, 2023
🌐 Local tile server for viewing geospatial raster files with ipyleaflet or folium

🌐 Local Tile Server for Geospatial Rasters Need to visualize a rather large (gigabytes) raster you have locally? This is for you. A Flask application

Bane Sullivan 192 Jan 04, 2023
Blender addons to make the bridge between Blender and geographic data

Blender GIS Blender minimal version : 2.8 Mac users warning : currently the addon does not work on Mac with Blender 2.80 to 2.82. Please do not report

5.9k Jan 02, 2023
Solving the Traveling Salesman Problem using Self-Organizing Maps

Solving the Traveling Salesman Problem using Self-Organizing Maps This repository contains an implementation of a Self Organizing Map that can be used

Diego Vicente 3.1k Dec 31, 2022
gjf: A tool for fixing invalid GeoJSON objects

gjf: A tool for fixing invalid GeoJSON objects The goal of this tool is to make it as easy as possible to fix invalid GeoJSON objects through Python o

Yazeed Almuqwishi 91 Dec 06, 2022
Manipulation and analysis of geometric objects

Shapely Manipulation and analysis of geometric objects in the Cartesian plane. Shapely is a BSD-licensed Python package for manipulation and analysis

3.1k Jan 03, 2023
Geocode rows in a SQLite database table

Geocode rows in a SQLite database table

Chris Amico 225 Dec 08, 2022
Program that shows all the details of the given IP address. Build with Python and ipinfo.io API

ip-details This is a program that shows all the details of the given IP address. Build with Python and ipinfo.io API Usage To use this program, run th

4 Mar 01, 2022
A package built to support working with spatial data using open source python

EarthPy EarthPy makes it easier to plot and manipulate spatial data in Python. Why EarthPy? Python is a generic programming language designed to suppo

Earth Lab 414 Dec 23, 2022
A Python framework for building geospatial web-applications

Hey there, this is Greppo... A Python framework for building geospatial web-applications. Greppo is an open-source Python framework that makes it easy

Greppo 304 Dec 27, 2022
WIP: extracting Geometry utilities from datacube-core

odc.geo This is still work in progress. This repository contains geometry related code extracted from Open Datacube. For details and motivation see OD

Open Data Cube 34 Jan 09, 2023
A ninja python package that unifies the Google Earth Engine ecosystem.

A Python package that unifies the Google Earth Engine ecosystem. EarthEngine.jl | rgee | rgee+ | eemont GitHub: https://github.com/r-earthengine/ee_ex

47 Dec 27, 2022
Replace MSFS2020's bing map to google map

English verison here 中文 免责声明 本教程提到的方法仅用于研究和学习用途。我不对使用、拓展该教程及方法所造成的任何法律责任和损失负责。 背景 微软模拟飞行2020的地景使用了Bing的卫星地图,然而卫星地图比较老旧,很多地区都是几年前的图设置直接是没有的。这种现象在全球不同地区

hesicong 272 Dec 24, 2022
Expose a GDAL file as a HTTP accessible on-the-fly COG

cogserver Expose any GDAL recognized raster file as a HTTP accessible on-the-fly COG (Cloud Optimized GeoTIFF) The on-the-fly COG file is not material

Even Rouault 73 Aug 04, 2022
Calculate the area inside of any GeoJSON geometry. This is a port of Mapbox's geojson-area for Python

geojson-area Calculate the area inside of any GeoJSON geometry. This is a port of Mapbox's geojson-area for Python. Installation $ pip install area U

Alireza 87 Dec 14, 2022