-
Notifications
You must be signed in to change notification settings - Fork 0
Description
I have made a number of fun changes ...
-
Made repo public
-
Added badge icons to the readme. This includes codecoverage with a really cool summary here!
-
Implemented unit tests in a workflow .github/workflows/test.yml
-
We now push to PyPi so we can do
pip install mapmanagercore.
This is done in a .github/workflows/publish_to_pypi.yml -
Removed large files like ch1 and ch2 .tiff and now use Pooch to lazily download them from a new Data repo
- Problem with GitHub large file storage (LFS) is that PyPi does not like anything larger than 100 MB (it defeats the whole purpose of GitHub LFS).
- We now lazily download files from the new MapManagerCore-Data repo using Pooch.
- This download is done in examples/example.ipynb and seems to work great.
- This Pooch download is also done in the /tests when necessary. See tests/test_gen_example_notebook
How to use mapmanagercore.data
The new code to lazily download data is in mapmanagercore.data
Example usage to get a points file is as follows. The getPointsFile() function downloads a file from MapManagerCore-Data and provides a local file path to that downloaded file.
Because Pooch is great, this file will (in theory) only be downloaded once. The second time you request it, it will use a local copy! Another reason pooch is great is that it all seems to work in GitHub workflows when we run, for example, unit tests!
import pandas as pd
from mapmanagercore.data import getPointsFile
df = pd.read_csv(getPointsFile())
print(df.head())Here is the full source ...
def getPointsFile():
"""Download and get path to points csv.
"""
urlPoints = 'https://github.com/mapmanager/MapManagerCore-Data/raw/main/data/rr30a_s0u/points.csv'
pointsPath = pooch.retrieve(
url=urlPoints,
known_hash=None
)
return pointsPath
def getLinesFile():
urlLines = 'https://github.com/mapmanager/MapManagerCore-Data/raw/main/data/rr30a_s0u/line_segments.csv'
linePath = pooch.retrieve(
url=urlLines,
known_hash=None
)
return linePath
def getTiffChannel_1():
urlCh1 = 'https://github.com/mapmanager/MapManagerCore-Data/raw/main/data/rr30a_s0u/t0/rr30a_s0_ch1.tif'
ch1Path = pooch.retrieve(
url=urlCh1,
known_hash=None
)
return ch1Path
def getTiffChannel_2():
urlCh2 = 'https://github.com/mapmanager/MapManagerCore-Data/raw/main/data/rr30a_s0u/t0/rr30a_s0_ch2.tif'
ch2Path = pooch.retrieve(
url=urlCh2,
known_hash=None,
)
return ch2Path