Skip to content

Conversation

@exequeryphil
Copy link
Contributor

This is a set of python scripts that might quickly provide some basic data for the many missing models (largely by scraping Huggingface). I guess I could've just pushed up the new YAML files, but we might want to run this again at some point, or make some minor adjustments to the operations.

Try it out by pulling down locally, installing the python dependencies in tools-py and then running the interactive script:

batch_scrape_missing.sh

It should compare our current models to the most active models on Huggingface, make a list of which to gather, then gather the data into YAML files in the ./models folder. There is some additional logic for determining the correct repository value.

Signed-off-by: Phil Williams <phil.williams@ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant