Releases: natcap/taskgraph
Releases · natcap/taskgraph
0.11.2
0.11.1
0.11.1 (2023-10-27)
- Adding
pyproject.tomlfor our build definitions. - Python 3.6 has reached end-of-life and is no longer maintained, so it has
been removed from the automated tests. - Python 3.7 has reached end-of-life and is no longer maintained, so it has
been removed from automated tests. - Python 3.11 has been released, so
taskgraphis now tested against this
new version of the language. - Python 3.12 has been released, so
taskgraphis now tested against this
new version of the language.
0.11.0
0.11.0 (2021-10-12)
- Testing against python 3.10 in github actions and officially noting support
for 3.10 insetup.py. - Testing against python 3.9 in github actions and noting support in
setup.py. - Fixed an issue where exceptions raised during execution where the task
completed beforeTaskGraph.join()was called would not be raised. Now,
if a task raises an exception, its exception will always be raised when
eitherTask.join()andTaskGraph.join()is called. - Fixed an issue where tasks with
hash_algorithm='sizetimestamp'would,
under certain conditions, fail to re-execute when they should. This only
occurred when a graph writing the same amount of , but possibly different,
data is executed successively, with less than about 1.5 seconds between
task executions. - After many years with the Natural Capital Project, Rich Sharp has stepped
down from the Project and as the maintainer oftaskgraph. James
Douglass is taking his place, and this change is now reflected in
setup.py. - Fixes an issue that causes an
EOFErrororBrokenPipeErrorto occur
when theTaskGraphterminates. - Updated the
taskgraphexample in the README for the latest API changes
and to clarify the need forif __name__ == '__main__': - Fixed an issue that could cause the
TaskGraphobject to hang if
duplicateTaskobjects were created. - Fixed an issue that was causing TaskGraph to ignore a changed
hash_algorithmif the TaskGraph was created on one run, was
deconstructed, then restarted. If the user chose a different hash, TaskGraph
would use the hash that the target file was originally hashed under rather
than the new algorithm. - Removed
copy_duplicate_artifactandhardlink_allowedparameters
and functionality from TaskGraph. This is to address a design error that
TaskGraph is not well suited for caching file results to avoid
recomputation. Rather than add additional complexity around the limitations
of this feature it is being removed to guide a design toward a standalone
cache library if needed.
0.10.3
0.10.3 (2021-01-29)
- Fixed issue that could cause combinatorial memory usage leading to poor
runtime orMemoryErrorif a dictionary were passed that had thousands
of elements. - Fixed issue that would cause
TaskGraphto not recognize a directory
that was meant to be ignored and in some cases causeTasks to
unnecessarily reexecute.
0.10.2
- Fixed an issue that would raise an exception when del was deconstructing a taskgraph object and a thread join() would cause a deadlock.
0.10.1
- Fixed an issue that would ignore the state of a transient_run flag if
a previous Task run had run it with that flag set to False. - Removed a limit on the number of times TaskGraph can attempt to update
its database up to 5 minutes of continuous failures. This is to address
expected issues when many parallel threads may compete for an update.
Relevant information about why the database update fails is logged. - Fixed an issue where the logging queue would always report an exception
even if the logging thread shut down correctly. - Fixed an issue that would raise an exception when del was
deconstructing a taskgraph object and a thread join() would cause a
deadlock.
0.10.0
Released Aug 25, 2020.
0.10.0 (2020-08-25)
- Fixed several race conditions that could cause the
TaskGraphobject to
hang on an otherwise ordinary termination. - Changed logging level to "INFO" on cases where the taskgraph was not
precalculated since it's an expected path of execution inTaskGraph. - Adding a
hardlink_allowedparameter toadd_taskthat allows the
attempt to hardlink a file in a case where acopy_artifact=Truemay
permit one. This will save on disk space as well as computation time
if large files are not needed to copy. - Adding a
store_resultflag toadd_taskthat conditionally stores
thefuncresult in the database for later.get. This was added to
guard against return types that were not picklable and would otherwise
cause an exception when being executed normally. - Fixed issue that would cause the logger thread to continue reporting status
after all tasks were complete and the graph was closed.
0.9.1
0.9.1 (2020-06-04)
- Fixed issue that would cause an infinite loop if a
TaskGraphobject were
created with a database from an incompatible previous version. Behavior now
is to log the issue, delete the old database, and create a new compatible
one. - Fixed issue that would cause some rare infinite loops if
TaskGraphwere
to fail due to some kinds of task exceptions. - Adding open source BSD-3-Clause license.
0.9.0
0.9.0 (2020-03-05)
- Updating primary repository URL to GitHub.
- Adding support for Python 3.8.
- Removing the
EncapsulatedOpabstract class. In practice the development
loop that encouraged the use ofEncapsulatedOpis flawed and can lead to
design errors. - Removing unnecessary internal locks which will improve runtime performance of
processing many small Tasks. - Refactor to support separate TaskGraph objects that use the same database.
- Removed the
n_retriesparameter fromadd_task. Users are recommended
to handle retries within functions themselves. - Added a
hash_target_filesflag toadd_taskthat when set to False,
causes TaskGraph to only note the existence of target files after execution
or as part of an evaluation to determine if the Task was precalculated.
This is useful for operations that initialize a file but subsequent runs of
the program modify it such as a new database or a downloaded file. - Fixed an issue on the monitor execution thread that caused shutdown of a
TaskGraph object to be delayed up to the amount of delay in the monitor
reporting update. - Added a
.get()function forTaskobjects that returns the result of
the respectivefunccall. This value is cached in the TaskGraph database
and hence can be used to avoid repeated execution. Note the addition of this
function changes the functionality of callingadd_taskwith no target
path list. In previous versions the Task would execute once per TaskGraph
instance, now successiveTaskobjects with the same execution signature
will use cached results. - To support the addition of the
.get()function atransient_run
parameter is added toadd_taskthat causes TaskGraph to avoid
recording a completedTaskeven if the execution hash would have been
identical to a previously completed run where the target artifacts still
existed.