-
Notifications
You must be signed in to change notification settings - Fork 74
/
Copy pathtar_destroy.Rd
186 lines (175 loc) · 7.83 KB
/
tar_destroy.Rd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/tar_destroy.R
\name{tar_destroy}
\alias{tar_destroy}
\title{Destroy the data store.}
\usage{
tar_destroy(
destroy = c("all", "cloud", "local", "meta", "process", "progress", "objects",
"scratch", "workspaces", "user"),
batch_size = 1000L,
verbose = TRUE,
ask = NULL,
script = targets::tar_config_get("script"),
store = targets::tar_config_get("store")
)
}
\arguments{
\item{destroy}{Character of length 1, what to destroy. Choices:
\itemize{
\item \code{"all"}: entire data store (default: \verb{_targets/})
including cloud data, as well as download/upload scratch files.
\item \code{"cloud"}: cloud data, including metadata, target object data
from targets with \code{tar_target(..., repository = "aws")},
and workspace files saved on the cloud.
Also deletes temporary staging files in
\code{file.path(tempdir(), "targets")}
that may have been accidentally left over from incomplete
uploads or downloads.
\item \code{"local"}: all the local files in the data store but nothing
on the cloud.
\item \code{"meta"}: metadata file at \code{meta/meta} in the
data store, which invalidates all the targets but keeps the data.
\item \code{"process"}: progress data file at
\code{meta/process} in the data store, which resets the metadata
of the main process.
\item \code{"progress"}: progress data file at
\code{meta/progress} in the data store,
which resets the progress tracking info.
\item \code{"objects"}: all the target
return values in \verb{objects/} in the data
store but keep progress and metadata.
Dynamic files are not deleted this way.
\item \code{"scratch"}: temporary files in saved during \code{\link[=tar_make]{tar_make()}} that should
automatically get deleted except if R crashed.
\item \code{"workspaces"}: compressed lightweight files locally saved
to the \verb{workspaces/} folder
in the data store with the saved workspaces of targets.
Does not delete workspace files on the cloud. For that,
consider \code{destroy = "all"} or \code{destroy = "cloud"}.
See \code{\link[=tar_workspace]{tar_workspace()}} for details.
\item \code{"user"}: custom user-supplied files in the \verb{user/} folder in the
data store.
}}
\item{batch_size}{Positive integer between 1 and 1000,
number of target objects to delete
from the cloud with each HTTP API request.
Currently only supported for AWS.
Cannot be more than 1000.}
\item{verbose}{Logical of length 1, whether to print console messages
to show progress when deleting each batch of targets from each
cloud bucket. Batched deletion with verbosity is currently only supported
for AWS.}
\item{ask}{Logical of length 1, whether to pause with a menu prompt
before deleting files. To disable this menu, set the \code{TAR_ASK}
environment variable to \code{"false"}. \code{usethis::edit_r_environ()}
can help set environment variables.}
\item{script}{Character of length 1, path to the
target script file. Defaults to \code{tar_config_get("script")},
which in turn defaults to \verb{_targets.R}. If the script does not exist,
then cloud metadata will not be deleted.}
\item{store}{Character of length 1, path to the
\code{targets} data store. Defaults to \code{tar_config_get("store")},
which in turn defaults to \verb{_targets/}.
When you set this argument, the value of \code{tar_config_get("store")}
is temporarily changed for the current function call.
See \code{\link[=tar_config_get]{tar_config_get()}} and \code{\link[=tar_config_set]{tar_config_set()}} for details
about how to set the data store path persistently
for a project.}
}
\value{
\code{NULL} (invisibly).
}
\description{
Destroy the data store written by the pipeline.
}
\details{
The data store is a folder created by \code{\link[=tar_make]{tar_make()}}
(or \code{\link[=tar_make_future]{tar_make_future()}} or \code{\link[=tar_make_clustermq]{tar_make_clustermq()}}).
The details of the data store are explained at
\url{https://books.ropensci.org/targets/data.html#local-data-store}.
The data store folder contains the output data
and metadata of the targets in the pipeline. Usually,
the data store is a folder called \verb{_targets/}
(see \code{\link[=tar_config_set]{tar_config_set()}} to customize), and it may
link to data on the cloud if you used AWS or GCP
buckets. By default, \code{tar_destroy()} deletes the entire
\verb{_targets/} folder (or wherever the data store is located),
including custom user-supplied files in \verb{_targets/user/},
as well as any cloud data that the pipeline uploaded.
See the \code{destroy} argument to customize this behavior
and only delete part of the data store, and see functions like
\code{\link[=tar_invalidate]{tar_invalidate()}}, \code{\link[=tar_delete]{tar_delete()}}, and \code{\link[=tar_prune]{tar_prune()}} to remove
information pertaining to some but not all targets in the pipeline.
After calling \code{tar_destroy()} with default arguments,
the entire data store is gone, which means all the output data from
previous runs of the pipeline is gone (except for
input/output files tracked with \code{tar_target(..., format = "file")}).
The next run of the pipeline will start from scratch,
and it will not skip any targets.
}
\section{Storage access}{
Several functions like \code{tar_make()}, \code{tar_read()}, \code{tar_load()},
\code{tar_meta()}, and \code{tar_progress()} read or modify
the local data store of the pipeline.
The local data store is in flux while a pipeline is running,
and depending on how distributed computing or cloud computing is set up,
not all targets can even reach it. So please do not call these
functions from inside a target as part of a running
pipeline. The only exception is literate programming
target factories in the \code{tarchetypes} package such as \code{tar_render()}
and \code{tar_quarto()}.
}
\section{Cloud target data versioning}{
Some buckets in Amazon S3 or Google Cloud Storage are "versioned",
which means they track historical versions of each data object.
If you use \code{targets} with cloud storage
(\url{https://books.ropensci.org/targets/cloud-storage.html})
and versioning is turned on, then \code{targets} will record each
version of each target in its metadata.
Functions like \code{\link[=tar_read]{tar_read()}}
and \code{\link[=tar_load]{tar_load()}} load the version recorded in the local metadata,
which may not be the same as the "current" version of the
object in the bucket. Likewise, functions \code{\link[=tar_delete]{tar_delete()}}
and \code{\link[=tar_destroy]{tar_destroy()}} only remove
the version ID of each target as recorded in the local
metadata.
If you want to interact with the \emph{latest} version of an object
instead of the version ID recorded in the local metadata,
then you will need to delete the object from the metadata.
\enumerate{
\item Make sure your local copy of the metadata is current and
up to date. You may need to run \code{\link[=tar_meta_download]{tar_meta_download()}} or
\code{\link[=tar_meta_sync]{tar_meta_sync()}} first.
\item Run \code{\link[=tar_unversion]{tar_unversion()}} to remove the recorded version IDs of
your targets in the local metadata.
\item With the version IDs gone from the local metadata,
functions like \code{\link[=tar_read]{tar_read()}} and \code{\link[=tar_destroy]{tar_destroy()}} will use the
\emph{latest} version of each target data object.
\item Optional: to back up the local metadata file with the version IDs
deleted, use \code{\link[=tar_meta_upload]{tar_meta_upload()}}.
}
}
\examples{
if (identical(Sys.getenv("TAR_EXAMPLES"), "true")) { # for CRAN
tar_dir({ # tar_dir() runs code from a temp dir for CRAN.
tar_script({
library(targets)
library(tarchetypes)
list(tar_target(x, 1 + 1))
})
tar_make() # Creates the _targets/ data store.
tar_destroy()
print(file.exists("_targets")) # Should be FALSE.
})
}
}
\seealso{
Other clean:
\code{\link{tar_delete}()},
\code{\link{tar_invalidate}()},
\code{\link{tar_prune}()},
\code{\link{tar_prune_list}()},
\code{\link{tar_unversion}()}
}
\concept{clean}