Files and Git
Read, write and watch sandbox files, and read and change git repositories, with @tektona/sdk.
sandbox.files and sandbox.git work with the file system of a sandbox. Every
operation runs as the image user by default. A path is absolute, or relative to the
sandbox working directory. ~ and ~/… name the home directory of the user that
the operation runs as. See Files and Git for the access rules and
the limits.
The SDK sends these requests to the access endpoint
of the sandbox, not to apiUrl, with the API key of the client. It reads the
endpoint from the sandbox once. Without an API key, a files or git call throws
AuthenticationError.
Files
const sandbox = await tek.sandbox.get('01J9F3ZK7QW4B2Y8N5T1QAZ6RX');
const { entries } = await sandbox.files.list('src', { depth: 2 });
for await (const entry of sandbox.files.listAll('.', { depth: 32, perDirLimit: 1000 })) {
console.log(entry.path);
}
const { type, size, etag: currentEtag } = await sandbox.files.stat('src/main.go');
const { bytes } = await sandbox.files.read('logo.png');
const { text, etag } = await sandbox.files.readText('src/main.go'); // throws on bytes that are not UTF-8
await sandbox.files.write('src/main.go', text + '\n// done\n', { ifMatch: etag });
await sandbox.files.write('notes.md', '# Notes\n', { createOnly: true });
await sandbox.files.move('notes.md', 'docs/notes.md', { overwrite: false });
await sandbox.files.remove('build', { recursive: true });
await sandbox.files.makeDir('build/out', { mode: 0o700 });
await sandbox.files.write('run.sh', '#!/bin/sh\necho ok\n', { mode: 0o755 });
await sandbox.files.setMode('run.sh', 0o700);| Method | Notes |
|---|---|
list(path, { depth, limit, perDirLimit, exclude, gitignore, cursor }) | One page, depth first: each folder is followed by its entries, in byte order of the names. depth defaults to 1, limit to 5000. When more entries remain, pagination.next_cursor is set; pass it as cursor with the same other options. A cursor of another listing is InvalidArgumentError. Symlinks are listed, not followed. exclude skips matching folders and files; gitignore skips what git ignores and needs write access. perDirLimit keeps the first entries of each folder, so that one large folder does not fill the pages. truncated and truncated_paths name the folders that miss entries that no page returns: a perDirLimit cut, a folder of more than 100,000 entries, or a folder that you cannot read. A file path lists the file itself, with no entries. A path under a file is 400 not_a_directory. |
listAll(path, { depth, limit, perDirLimit, exclude, gitignore }) | Every entry, page by page: the entry at path first, then each entry under it once. limit sets the page size. A file added during the walk shows up when it sorts after the position of the walk, and is missed when it sorts before. |
stat(path) | The entry at path, as list(path, { depth: 0 }) returns it, for a file, a folder or a symlink (not followed). A regular file also has an etag. |
etag(path) | The ETag of a regular file, without its bytes. Every regular file has one; treat it as opaque. |
read(path, { ifNoneMatch }) | { bytes, etag } for a file of any size. With ifNoneMatch, a file that still has that ETag returns { notModified: true } and no bytes. |
readText(path) | { text, etag? }. |
write(path, data, { ifMatch, createOnly, mode }) | Stream a string, Uint8Array, Blob or ReadableStream of any size. mode (for example 0o755) is set before the file replaces the target. Returns { entry, etag, created }; created is true for a new file. An ifMatch key with the value undefined throws InvalidArgumentError, so a write never loses its check. |
readStream(path) | A ReadableStream of the file. It rejects a short body. |
readStream(path, { offset, ifRange }) | Resume at offset. Returns { stream, partial, etag }: append when partial, else the stream is the whole file. |
remove(path, { recursive }) | Without recursive, deletes a file, a symlink or an empty directory, like rm -d. A directory that is not empty needs recursive, else 409 directory_not_empty. |
move(from, to, { overwrite }) | Returns the entry at to. Do not retry a move blindly: the first try can have moved the file. |
makeDir(path, { mode }) | Creates missing parents too. Returns { entry, created }; created is false for an existing directory. |
setMode(path, mode) | Like chmod; follows a symlink. Returns the entry. |
setOwner(path, { owner, group }) | Like chown; names or numbers, follows a symlink. Returns the entry. |
Pass a mode as an octal literal such as 0o755; the API carries it as a
decimal integer. An entry has uid, gid and, when the sandbox has names for
them, owner and group.
A write uses a temporary file. A failed upload leaves the target unchanged.
When write returns, the file is synced to disk. A failed sync fails the
write and keeps the old file.
The new file has a new inode; hard links keep the old bytes. ifMatch protects
against other API writes, but not against processes in the sandbox.
A sandbox has at most 32 open transfers. More return 429 too_many_transfers.
A transfer stops when it moves less than 256 KiB in a
minute. Every regular file has an ETag, so ifMatch and ifNoneMatch work at
any size. Treat the ETag as opaque: do not compare it with a hash.
const res = await fetch('https://example.com/dataset.tar');
await sandbox.files.write('data/dataset.tar', res.body!);
const stream = await sandbox.files.readStream('out/report.pdf');
// A whole folder as tar.gz, without build output.
const archive = await sandbox.files.downloadArchive('app', { format: 'tar.gz', exclude: ['node_modules', '.git'] });downloadArchive also takes a list of paths; each goes into the archive under its base name. It returns a stream that rejects when the download breaks.
Write many files in one request, or upload an archive:
await sandbox.files.writeFiles([
{ path: 'app/src/index.ts', data: source },
{ path: 'app/bin/run', data: script, mode: 0o755 },
]);
await sandbox.files.uploadArchive('app', archiveBytes, { format: 'tar.gz' });Hand a file or a folder to a client without credentials:
const { url, expires_at } = await sandbox.files.signedUrl('dist/app.zip', { method: 'GET', expiresIn: 300 });
const upload = await sandbox.files.signedUrl('inbox/report.pdf', { method: 'PUT' });
await fetch(upload.url, { method: 'PUT', body: pdf }); // uploads once
const folder = await sandbox.files.signedUrl(['app', 'docs'], { method: 'GET', format: 'tar.gz', exclude: ['node_modules'] });signedUrl returns { url, method, expires_at, single_use }. With format, a
GET URL downloads an archive, as downloadArchive does. A signed URL never
resumes a paused sandbox.
writeFiles packs the files into one tar for their common directory. A file
without mode keeps the mode of the file that it replaces, or gets 0o644
when it is new. Each
file is replaced atomically, the set is not: a failure keeps the files before
it.
Edit files as root
Every sandbox.files method takes a user option, a user name or a uid. The
operation then runs as that user. Use it for files that the image user cannot
change:
const { text, etag } = await sandbox.files.readText('/etc/hosts', { user: 'root' });
await sandbox.files.write('/etc/hosts', text + '127.0.0.1 api.local\n', { user: 'root', ifMatch: etag });An operation with user needs permission to operate the sandbox, also a read.
Paused sandboxes
A call on a paused sandbox throws SandboxPausedError. Set resume: true on
a sandbox.files or sandbox.git call to resume the sandbox first. watch
and signedUrl never resume a sandbox.
const { entries } = await sandbox.files.list('src', { resume: true });Errors
A refused operation throws an ApiError subclass. error.code holds the
reason, for example precondition_failed, too_large or
directory_not_empty. error.retryable is true when the same call can
succeed later without a change: busy, a 429 and a 503.
| Class | When |
|---|---|
FileNotFoundError | 404 not_found: the path does not exist. |
SandboxNotFoundError | 404 sandbox_not_found. |
SandboxPausedError | 409 sandbox_paused. |
PreconditionFailedError | 412 precondition_failed: ifMatch or createOnly failed. |
GitError | A git_* code, for example git_failed or git_identity_missing. |
UnavailableError | 503: the sandbox did not answer. Retry later. |
Result shapes
A method returns the body of the API response, with snake_case keys. A method
unwraps the body only when the body holds entry and nothing else: move,
setMode and setOwner return the entry. A method that lists returns one page
with pagination; its …All twin yields every item.
Git
Git operations need permission to operate the sandbox. Git always runs as the
image user, so the git config and the hooks of that user apply. Every
sandbox.git method takes the work tree in opts.path, default .. Only
clone and init take the path as an argument.
const status = await sandbox.git.status();
const diff = await sandbox.git.diff({ base: 'main' });
const summary = await sandbox.git.diffSummary({ path: 'app', base: 'main' });
for (const f of diff.files) {
const before = f.old_oid ? await sandbox.git.blob(f.old_oid) : new Uint8Array();
const after = f.status === 'deleted' ? new Uint8Array() : (await sandbox.files.read(`${diff.root}/${f.path}`)).bytes;
showDiff(f.path, before, after);
}
for await (const path of sandbox.git.filesAll()) console.log(path);diff.files has a lines object for each file with known changed-line counts.
The value is null if the count is unknown.
summary.files.binary counts files without a known line count.
summary.lines adds only the known changed-line counts.
patch returns every change as one patch for git apply -p1, untracked and
binary files included. Write bytes to a file, not patch: a file that is not
UTF-8 keeps its exact bytes only there.
const { bytes, truncated } = await sandbox.git.patch({ files: ['src/login.ts'] });In a very large work tree, status, diff, diffSummary and patch return
truncated: true and a part of the files instead of an error. No call returns
the rest.
log, branches, files and ignored return one page with pagination.
Pass pagination.next_cursor as cursor for the next page, or use logAll,
branchesAll, filesAll and ignoredAll.
Change a repository:
const { root } = await sandbox.git.clone('https://github.com/acme/app.git', 'app');
await sandbox.git.createBranch('fix-login', { path: root, checkout: true });
await sandbox.git.stage(['src/login.ts'], { path: root });
const { oid } = await sandbox.git.commit('Fix the login form', {
path: root,
authorName: 'Agent',
authorEmail: 'agent@example.com',
});
await sandbox.git.push({ path: root, branch: 'fix-login', setUpstream: true });
const { commits } = await sandbox.git.log({ path: root, limit: 20 });checkout, unstage, discard, deleteBranch, fetch, pull, init,
reset, restore, remotes, addRemote, removeRemote, getConfig and
setConfig work the same way. createBranch and addRemote return what they
create. A missing path throws FileNotFoundError. A missing remote in
removeRemote and an unknown object in blob throw NotFoundError. getConfig returns undefined for a key that is not set. getConfig
and setConfig take only user.name, user.email, core.autocrlf,
pull.rebase and init.defaultBranch. A commit without an author and without
user.name and user.email in the sandbox throws GitError with the code
git_identity_missing. A commit with nothing to commit throws with
nothing_to_commit. A command that git refuses throws GitError with
git_failed. Do not retry reset, pull or push blindly: the first try can
have run.
To keep a change view current, call status 300 ms after the last changes
frame of a watch, and every 30 seconds while the view is visible.
Watch
watch returns an async iterable of ready, changes, resync and a last
end event:
import { FilesWatchLostError } from '@tektona/sdk';
async function follow(): Promise<void> {
let delay = 500;
for (;;) {
const watch = sandbox.files.watch('src', { recursive: true, exclude: ['node_modules'] });
try {
for await (const ev of watch) {
if (ev.type === 'ready' || ev.type === 'resync') {
loadTree(await Array.fromAsync(sandbox.files.listAll('src', { depth: 4, exclude: ['node_modules'] })));
delay = 500;
} else if (ev.type === 'changes') {
applyChanges(ev.changes);
} else {
console.log('watch ended:', ev.reason);
return;
}
}
return;
} catch (e) {
if (!(e instanceof FilesWatchLostError)) throw e; // For example, access was revoked.
await new Promise((resolve) => setTimeout(resolve, delay));
delay = Math.min(delay * 2, 30_000);
}
}
}The iteration throws FilesWatchLostError when a seq is skipped, a frame is
not valid, or the socket closes without end. Call watch again and list
again. watch.close() stops the watch without an error.
A watch that fails before ready throws the error of the cause instead, for
example SandboxPausedError or AuthorizationError. A watch never resumes a
paused sandbox.
reason is expired, server_shutdown, or connection_lost.
After a lost watch, wait up to 30 seconds before you reopen it.
The SDK does not reconnect for you.