stephenseo/SimpleArchiver
1
0
Fork 0
Code Issues 2 Pull requests Projects 1 Releases 6 Packages Wiki Activity Actions

Compare commits

base: stephenseo:main
Branches Tags
stephenseo:main
stephenseo:dev
stephenseo:windows
stephenseo:1.5
stephenseo:1.4
stephenseo:1.3
stephenseo:1.2
stephenseo:1.1
stephenseo:1.0
stephenseo:stream_03
..
compare: stephenseo:1.0
Branches Tags
stephenseo:dev
stephenseo:main
stephenseo:windows
stephenseo:1.5
stephenseo:1.4
stephenseo:1.3
stephenseo:1.2
stephenseo:1.1
stephenseo:1.0
stephenseo:stream_03

No commits in common. "main" and "1.0" have entirely different histories.
main ... 1.0

16 changed files with 35 additions and 372 deletions
Show stats Expand all files Collapse all files

1
CMakeLists.txt
View file

@ -87,5 +87,4 @@ add_executable(test_simplearchiver
src/algorithms/linear_congruential_gen.c
src/data_structures/linked_list.c
src/data_structures/hash_map.c
src/data_structures/priority_heap.c
)

56
Changelog.md
View file

@ -1,56 +0,0 @@
# Changelog
## Upcoming Changes
## Version 1.5
Previous file-format-v1 implementation of "safe links" still created a symlink
if a relative or absolute link existed in the file. This version fixes this, and
prevents invalid symlinks from being created. (This check is only done if the
bit-flag is set in the file as mentioned in the file-format spec for v1 files.)
## Version 1.4
Do "safe links" behavior by default: symlinks pointing to outside of archived
files (or invalid symlinks) should not be included in the archive, unless if the
option "--no-safe-links" is specified. This is supported in both v0 and v1 file
formats.
## Version 1.3
Prevent `simplearchiver` from busy-waiting during non-blocking IO by sleeping
in "EWOULDBLOCK" conditions. This results in less consumed cpu time by the
process, especially during compression.
## Version 1.2
Proper handling of Ctrl+C (SIGINT). This prevents temporary files from
persisting by doing a proper cleanup before stopping the program.
## Version 1.1
More robust handling of de/compression process (handling SIGPIPE).
By default files are now pre-sorted by size before placed into chunks.
Add option to NOT pre-sort files by size.
## Version 1.0
First release.
Features:
- Can specify any command as de/compressor when archiving.
- The commands must accept file data in stdin and output processed data to
stdout.
- Can specify any command as decompressor when extracting to override the
simple-archive's stored decompressor.
- Archives/compresses into chunks to reduce overhead by compressing per chunk
instead of per file.
- Chunk size can be tweaked by a parameter setting.
- Can archive without de/compressor to be compressed separately.
- Supports pre-version-1 simple archiver file format (version 0).
- Archives regular files and symlinks.
- Keeps track of files and symlink permissions.
- Keeps track of file UID and GID (only set if extracting as root).
- Can be set to ignore absolute paths for symlinks by parameter setting.

11
README.md
View file

@ -1,8 +1,7 @@
# Simple Archiver
This program ~~is not yet~~ ~~almost~~ basically finished! ~~Basic~~ Necessary
functionality is implemented and only ~~some advanced features are missing~~
some extra features are not yet implemented. You can track progress
This program ~~is not yet~~ almost finished! Basic functionality is implemented
and only some advanced features are missing. You can track progress
[here](https://git.seodisparate.com/stephenseo/SimpleArchiver/projects/3).
This program exists because I could not get `tar` or `ar` to compile with
@ -27,11 +26,9 @@ API calls.
--overwrite-create : allows overwriting an archive file
--overwrite-extract : allows overwriting when extracting
--no-abs-symlink : do not store absolute paths for symlinks
--no-safe-links : keep symlinks that link to outside archive contents
--temp-files-dir <dir> : where to store temporary files created when compressing (defaults to current working directory)
--write-version <version> : Force write version file format (default 1)
--chunk-min-size <bytes> : v1 file format minimum chunk size (default 4194304 or 4MiB)
--no-pre-sort-files : do NOT pre-sort files by size (by default enabled so that the first file is the largest)
-- : specifies remaining arguments are files to archive/extract
If creating archive file, remaining args specify files to archive.
If extracting archive file, remaining args specify files to extract.
@ -39,10 +36,6 @@ API calls.
Note that `--compressor` and `--decompressor` cmds must accept data from stdin
and return processed data to stdout.
## Changes
See the [Changelog](https://git.seodisparate.com/stephenseo/SimpleArchiver/src/branch/main/Changelog.md).
## LICENSE Information
Uses the [ISC License](https://choosealicense.com/licenses/isc/).

8
file_format.md
View file

@ -57,11 +57,6 @@ Following the file-count bytes, the following bytes are added for each file:
2. The second bit is "other execute permission".
3. The third bit is UNSET if relative links are preferred, and is SET
if absolute links are preferred.
4. The fourth bit is set if this file/symlink-entry is invalid and must
be skipped. Ignore following bytes after these 4 bytes bit-flags in
this specification and skip to the next entry; if marked invalid,
the following specification bytes for this file/symlink entry must
not exist.
3. The third byte.
1. Currently unused.
4. The fourth byte.
@ -133,9 +128,6 @@ Following the link-count bytes, the following bytes are added for each symlink:
2. The second byte.
1. The first bit is "other write permission".
2. The second bit is "other execute permission".
3. If this bit is set, then this entry is marked invalid. The link name
will be preserved in this entry, but the following link target paths
will be set to zero-length and will not be stored.
2. 2 bytes 16-bit unsigned integer "link name" in big-endian. This does not
include the NULL at the end of the string. Must not be zero.
3. X bytes of link-name (length defined by previous value). Is a NULL-terminated

0
test_files/file_format_1_example_0 → file_format_1_example_0
View file

0
test_files/file_format_1_example_1 → file_format_1_example_1
View file

0
test_files/invalid_file_format_0_example_0 → invalid_file_format_0_example_0
View file

0
test_files/invalid_file_format_0_example_1 → invalid_file_format_0_example_1
View file

0
test_files/invalid_file_format_1_example_0 → invalid_file_format_1_example_0
View file

0
test_files/invalid_file_format_1_example_1 → invalid_file_format_1_example_1
View file

0
test_files/invalid_file_format_1_example_2 → invalid_file_format_1_example_2
View file

307
src/archiver.c
View file

@ -38,7 +38,6 @@
#include <unistd.h>
#endif
#include "data_structures/priority_heap.h"
#include "helpers.h"
#define TEMP_FILENAME_CMP "%s%ssimple_archiver_compressed_%lu.tmp"
@ -51,21 +50,12 @@
SIMPLE_ARCHIVER_PLATFORM == SIMPLE_ARCHIVER_PLATFORM_MAC || \
SIMPLE_ARCHIVER_PLATFORM == SIMPLE_ARCHIVER_PLATFORM_LINUX
volatile int is_sig_pipe_occurred = 0;
volatile int is_sig_int_occurred = 0;
void handle_sig_pipe(int sig) {
if (sig == SIGPIPE) {
is_sig_pipe_occurred = 1;
}
}
void handle_sig_int(int sig) {
if (sig == SIGINT) {
is_sig_int_occurred = 1;
}
}
const struct timespec nonblock_sleep = {.tv_sec = 0, .tv_nsec = 1000000};
#endif
typedef struct SDArchiverInternalToWrite {
@ -114,25 +104,7 @@ void cleanup_temp_filename_delete(void ***ptrs_array) {
#endif
}
void cleanup_overwrite_filename_delete_simple(char **filename) {
if (filename && *filename) {
if ((*filename)[0] != 0) {
#if SIMPLE_ARCHIVER_PLATFORM == SIMPLE_ARCHIVER_PLATFORM_COSMOPOLITAN || \
SIMPLE_ARCHIVER_PLATFORM == SIMPLE_ARCHIVER_PLATFORM_MAC || \
SIMPLE_ARCHIVER_PLATFORM == SIMPLE_ARCHIVER_PLATFORM_LINUX
unlink(*filename);
#endif
}
free(*filename);
*filename = NULL;
}
}
int write_files_fn(void *data, void *ud) {
if (is_sig_int_occurred) {
return 1;
}
const SDArchiverFileInfo *file_info = data;
SDArchiverState *state = ud;
@ -285,17 +257,6 @@ int write_files_fn(void *data, void *ud) {
size_t read_count;
ssize_t ret;
while (!write_done || !read_done) {
if (is_sig_int_occurred) {
if (pipe_into_cmd[1] >= 0) {
close(pipe_into_cmd[1]);
pipe_into_cmd[1] = -1;
}
if (pipe_outof_cmd[0] >= 0) {
close(pipe_outof_cmd[0]);
pipe_outof_cmd[0] = -1;
}
return 1;
}
if (is_sig_pipe_occurred) {
fprintf(stderr,
"WARNING: Failed to write to compressor (SIGPIPE)! Invalid "
@ -313,7 +274,6 @@ int write_files_fn(void *data, void *ud) {
ret = write(pipe_into_cmd[1], write_buf, write_count);
if (ret == -1) {
if (errno == EAGAIN || errno == EWOULDBLOCK) {
nanosleep(&nonblock_sleep, NULL);
write_again = 1;
} else {
// Error during write.
@ -337,7 +297,6 @@ int write_files_fn(void *data, void *ud) {
simple_archiver_helper_cleanup_FILE(&file_fd);
write_done = 1;
close(pipe_into_cmd[1]);
pipe_into_cmd[1] = -1;
// fprintf(stderr, "write_done\n");
} else if (ferror(file_fd)) {
// Error during read file.
@ -367,11 +326,10 @@ int write_files_fn(void *data, void *ud) {
read_done = 1;
simple_archiver_helper_cleanup_FILE(&tmp_fd);
close(pipe_outof_cmd[0]);
pipe_outof_cmd[0] = -1;
// fprintf(stderr, "read_done\n");
} else if (ret == -1) {
if (errno == EAGAIN || errno == EWOULDBLOCK) {
nanosleep(&nonblock_sleep, NULL);
// Nop.
} else {
// Read error.
fprintf(stderr,
@ -390,8 +348,6 @@ int write_files_fn(void *data, void *ud) {
"WARNING: Failed to write to compressor (SIGPIPE)! Invalid "
"compressor cmd?\n");
return 1;
} else if (is_sig_int_occurred) {
return 1;
}
waitpid(compressor_pid, NULL, 0);
@ -649,10 +605,6 @@ int write_files_fn(void *data, void *ud) {
simple_archiver_list_free(&to_write);
if (is_sig_int_occurred) {
return 1;
}
// Write file.
fprintf(stderr, "Writing file: %s\n", file_info->filename);
char buf[SIMPLE_ARCHIVER_BUFFER_SIZE];
@ -787,39 +739,16 @@ int write_files_fn(void *data, void *ud) {
if (abs_path && (state->parsed->flags & 0x20) == 0 &&
!simple_archiver_hash_map_get(state->map, abs_path,
strlen(abs_path) + 1)) {
// Is not a filename being archived.
if ((state->parsed->flags & 0x80) != 0) {
// No safe links, set preference to absolute path.
fprintf(
stderr,
"NOTICE: abs_path exists, \"--no-abs-symlink\" not specified, "
"and link refers to file NOT in archive; preferring abs_path.\n");
((uint8_t *)temp_to_write->buf)[1] |= 0x4;
} else {
// Safe links, do not store symlink!
fprintf(stderr,
"WARNING: Symlink \"%s\" points to outside archive contents, "
"will not be stored! (Use \"--no-safe-links\" to disable this "
"behavior)\n",
file_info->filename);
((uint8_t *)temp_to_write->buf)[1] |= 0x8;
}
// Is not a filename being archived, set preference to absolute path.
fprintf(stderr,
"NOTICE: abs_path exists, \"--no-abs-symlink\" not specified, "
"and link refers to file NOT in archive; preferring abs_path.\n");
((uint8_t *)temp_to_write->buf)[1] |= 0x4;
}
// Store the 4 byte bit-flags for file.
simple_archiver_list_add(to_write, temp_to_write, free_internal_to_write);
if ((((uint8_t *)temp_to_write->buf)[1] & 0x8) != 0) {
// Skipped symlink.
simple_archiver_list_get(to_write, write_list_datas_fn, state->out_f);
simple_archiver_list_free(&to_write);
char format_str[64];
snprintf(format_str, 64, FILE_COUNTS_OUTPUT_FORMAT_STR_1, state->digits,
state->digits);
fprintf(stderr, format_str, ++(state->count), state->max);
return 0;
}
// Store the absolute and relative paths.
if (!abs_path) {
fprintf(stderr,
@ -907,10 +836,6 @@ int write_files_fn(void *data, void *ud) {
snprintf(format_str, 64, FILE_COUNTS_OUTPUT_FORMAT_STR_1, state->digits,
state->digits);
fprintf(stderr, format_str, ++(state->count), state->max);
if (is_sig_int_occurred) {
return 1;
}
return 0;
}
@ -1169,18 +1094,12 @@ int read_decomp_to_out_file(const char *out_filename, int in_pipe,
ssize_t read_ret;
size_t fwrite_ret;
while (written_amt < file_size) {
if (is_sig_pipe_occurred) {
fprintf(stderr, "ERROR: SIGPIPE while decompressing!\n");
return SDAS_INTERNAL_ERROR;
}
int ret = try_write_to_decomp(to_dec_pipe, chunk_remaining, in_f, read_buf,
read_buf_size, hold_buf, has_hold);
if (ret != SDAS_SUCCESS) {
return ret;
} else if (is_sig_pipe_occurred) {
fprintf(stderr, "ERROR: SIGPIPE while decompressing!\n");
return SDAS_INTERNAL_ERROR;
} else if (file_size - written_amt >= read_buf_size) {
}
if (file_size - written_amt >= read_buf_size) {
read_ret = read(in_pipe, read_buf, read_buf_size);
if (read_ret > 0) {
if (out_fd) {
@ -1211,7 +1130,6 @@ int read_decomp_to_out_file(const char *out_filename, int in_pipe,
} else {
if (errno == EAGAIN || errno == EWOULDBLOCK) {
// Non-blocking read from pipe.
nanosleep(&nonblock_sleep, NULL);
continue;
} else {
// Error.
@ -1251,7 +1169,6 @@ int read_decomp_to_out_file(const char *out_filename, int in_pipe,
} else {
if (errno == EAGAIN || errno == EWOULDBLOCK) {
// Non-blocking read from pipe.
nanosleep(&nonblock_sleep, NULL);
continue;
} else {
// Error.
@ -1552,7 +1469,6 @@ int symlinks_and_files_from_files(void *data, void *ud) {
SDArchiverLinkedList *symlinks_list = ptr_array[0];
SDArchiverLinkedList *files_list = ptr_array[1];
const char *user_cwd = ptr_array[2];
SDArchiverPHeap *pheap = ptr_array[3];
if (file_info->filename) {
if (file_info->link_dest) {
@ -1639,14 +1555,8 @@ int symlinks_and_files_from_files(void *data, void *ud) {
return 1;
}
file_info_struct->file_size = (uint64_t)ftell_ret;
if (pheap) {
simple_archiver_priority_heap_insert(
pheap, (int64_t)file_info_struct->file_size, file_info_struct,
free_internal_file_info);
} else {
simple_archiver_list_add(files_list, file_info_struct,
free_internal_file_info);
}
simple_archiver_list_add(files_list, file_info_struct,
free_internal_file_info);
}
}
@ -1678,8 +1588,6 @@ int files_to_chunk_count(void *data, void *ud) {
return 0;
}
int greater_fn(int64_t a, int64_t b) { return a > b; }
char *simple_archiver_error_to_string(enum SDArchiverStateReturns error) {
switch (error) {
case SDAS_SUCCESS:
@ -1706,8 +1614,6 @@ char *simple_archiver_error_to_string(enum SDArchiverStateReturns error) {
return "Failed to change current working directory";
case SDAS_INVALID_WRITE_VERSION:
return "Unsupported write version file format";
case SDAS_SIGINT:
return "Interrupt signal SIGINT recieved";
default:
return "Unknown error";
}
@ -1739,11 +1645,6 @@ void simple_archiver_free_state(SDArchiverState **state) {
int simple_archiver_write_all(FILE *out_f, SDArchiverState *state,
const SDArchiverLinkedList *filenames) {
#if SIMPLE_ARCHIVER_PLATFORM == SIMPLE_ARCHIVER_PLATFORM_COSMOPOLITAN || \
SIMPLE_ARCHIVER_PLATFORM == SIMPLE_ARCHIVER_PLATFORM_MAC || \
SIMPLE_ARCHIVER_PLATFORM == SIMPLE_ARCHIVER_PLATFORM_LINUX
signal(SIGINT, handle_sig_int);
#endif
switch (state->parsed->write_version) {
case 0:
return simple_archiver_write_v0(out_f, state, filenames);
@ -1863,10 +1764,6 @@ int simple_archiver_write_v0(FILE *out_f, SDArchiverState *state,
}
}
if (is_sig_int_occurred) {
return SDAS_SIGINT;
}
// Iterate over files in list to write.
state->count = 0;
state->max = filenames->count;
@ -1891,9 +1788,6 @@ int simple_archiver_write_v0(FILE *out_f, SDArchiverState *state,
state->digits);
fprintf(stderr, format_str, state->count, state->max);
if (simple_archiver_list_get(filenames, write_files_fn, state)) {
if (is_sig_int_occurred) {
return SDAS_SIGINT;
}
// Error occurred.
fprintf(stderr, "Error ocurred writing file(s) to archive.\n");
return SDAS_FAILED_TO_WRITE;
@ -1923,17 +1817,11 @@ int simple_archiver_write_v1(FILE *out_f, SDArchiverState *state,
SDArchiverLinkedList *symlinks_list = simple_archiver_list_init();
__attribute__((cleanup(simple_archiver_list_free)))
SDArchiverLinkedList *files_list = simple_archiver_list_init();
__attribute__((cleanup(simple_archiver_priority_heap_free)))
SDArchiverPHeap *files_pheap =
(state->parsed->flags & 0x40)
? simple_archiver_priority_heap_init_less_fn(greater_fn)
: NULL;
ptr_array = malloc(sizeof(void *) * 4);
ptr_array = malloc(sizeof(void *) * 3);
ptr_array[0] = symlinks_list;
ptr_array[1] = files_list;
ptr_array[2] = (void *)state->parsed->user_cwd;
ptr_array[3] = files_pheap;
if (simple_archiver_list_get(filenames, symlinks_and_files_from_files,
ptr_array)) {
@ -1942,22 +1830,6 @@ int simple_archiver_write_v1(FILE *out_f, SDArchiverState *state,
}
free(ptr_array);
if (files_pheap) {
while (files_pheap->size > 0) {
simple_archiver_list_add(files_list,
simple_archiver_priority_heap_pop(files_pheap),
free_internal_file_info);
}
simple_archiver_priority_heap_free(&files_pheap);
}
if (symlinks_list->count + files_list->count != filenames->count) {
fprintf(stderr,
"ERROR: Count mismatch between files and symlinks and files from "
"parser!\n");
return SDAS_INTERNAL_ERROR;
}
if (fwrite("SIMPLE_ARCHIVE_VER", 1, 18, out_f) != 18) {
return SDAS_FAILED_TO_WRITE;
}
@ -2079,25 +1951,11 @@ int simple_archiver_write_v1(FILE *out_f, SDArchiverState *state,
abs_path);
}
}
uint_fast8_t is_invalid = 0;
if (abs_path && (state->parsed->flags & 0x20) == 0 &&
!simple_archiver_hash_map_get(abs_filenames, abs_path,
strlen(abs_path) + 1)) {
// Is not a filename being archived.
if ((state->parsed->flags & 0x80) == 0) {
// Not a "safe link", mark invalid and continue.
is_invalid = 1;
fprintf(stderr,
"WARNING: \"%s\" points to outside of archived files (or is "
"invalid) and \"--no-safe-links\" not specified, will not "
"store abs/rel-links to this entry!\n",
(const char *)node->data);
} else {
// Safe links disabled, set preference to absolute path.
buf[0] |= 1;
}
// Is not a filename being archived, set preference to absolute path.
buf[0] |= 1;
}
// Get symlink stats for permissions.
@ -2140,11 +1998,6 @@ int simple_archiver_write_v1(FILE *out_f, SDArchiverState *state,
buf[0] = 0xFE;
buf[1] = 3;
#endif
if (is_invalid) {
buf[1] |= 4;
}
if (fwrite(buf, 1, 2, out_f) != 2) {
return SDAS_FAILED_TO_WRITE;
}
@ -2165,7 +2018,7 @@ int simple_archiver_write_v1(FILE *out_f, SDArchiverState *state,
return SDAS_FAILED_TO_WRITE;
}
if (abs_path && (state->parsed->flags & 0x20) == 0 && !is_invalid) {
if (abs_path && (state->parsed->flags & 0x20) == 0) {
len = strlen(abs_path);
if (len >= 0xFFFF) {
fprintf(stderr,
@ -2189,7 +2042,7 @@ int simple_archiver_write_v1(FILE *out_f, SDArchiverState *state,
}
}
if (rel_path && !is_invalid) {
if (rel_path) {
len = strlen(rel_path);
if (len >= 0xFFFF) {
fprintf(stderr,
@ -2220,10 +2073,6 @@ int simple_archiver_write_v1(FILE *out_f, SDArchiverState *state,
}
}
if (is_sig_int_occurred) {
return SDAS_SIGINT;
}
__attribute__((cleanup(simple_archiver_list_free)))
SDArchiverLinkedList *chunk_counts = simple_archiver_list_init();
@ -2291,9 +2140,6 @@ int simple_archiver_write_v1(FILE *out_f, SDArchiverState *state,
uint64_t chunk_count = 0;
for (SDArchiverLLNode *chunk_c_node = chunk_counts->head->next;
chunk_c_node != chunk_counts->tail; chunk_c_node = chunk_c_node->next) {
if (is_sig_int_occurred) {
return SDAS_SIGINT;
}
fprintf(stderr, "CHUNK %3lu of %3lu\n", ++chunk_count, chunk_counts->count);
// Write file count before iterating through files.
if (non_c_chunk_size) {
@ -2461,9 +2307,6 @@ int simple_archiver_write_v1(FILE *out_f, SDArchiverState *state,
int_fast8_t to_temp_finished = 0;
for (uint64_t file_idx = 0; file_idx < *((uint64_t *)chunk_c_node->data);
++file_idx) {
if (is_sig_int_occurred) {
return SDAS_SIGINT;
}
file_node = file_node->next;
if (file_node == files_list->tail) {
return SDAS_INTERNAL_ERROR;
@ -2478,10 +2321,6 @@ int simple_archiver_write_v1(FILE *out_f, SDArchiverState *state,
char hold_buf[SIMPLE_ARCHIVER_BUFFER_SIZE];
ssize_t has_hold = -1;
while (!to_comp_finished) {
if (is_sig_pipe_occurred) {
fprintf(stderr, "ERROR: SIGPIPE while compressing!\n");
return SDAS_INTERNAL_ERROR;
}
if (!to_comp_finished) {
// Write to compressor.
if (ferror(fd)) {
@ -2497,7 +2336,6 @@ int simple_archiver_write_v1(FILE *out_f, SDArchiverState *state,
// Non-blocking write.
has_hold = (int)fread_ret;
memcpy(hold_buf, buf, fread_ret);
nanosleep(&nonblock_sleep, NULL);
} else {
fprintf(
stderr,
@ -2524,7 +2362,6 @@ int simple_archiver_write_v1(FILE *out_f, SDArchiverState *state,
if (write_ret < 0) {
if (errno == EAGAIN || errno == EWOULDBLOCK) {
// Non-blocking write.
nanosleep(&nonblock_sleep, NULL);
} else {
return SDAS_INTERNAL_ERROR;
}
@ -2547,7 +2384,6 @@ int simple_archiver_write_v1(FILE *out_f, SDArchiverState *state,
if (read_ret < 0) {
if (errno == EAGAIN || errno == EWOULDBLOCK) {
// Non-blocking read.
nanosleep(&nonblock_sleep, NULL);
} else {
fprintf(stderr,
"ERROR: Reading from compressor, pipe read error!\n");
@ -2573,16 +2409,11 @@ int simple_archiver_write_v1(FILE *out_f, SDArchiverState *state,
// Finish writing.
if (!to_temp_finished) {
while (1) {
if (is_sig_pipe_occurred) {
fprintf(stderr, "ERROR: SIGPIPE while compressing!\n");
return SDAS_INTERNAL_ERROR;
}
ssize_t read_ret =
read(pipe_outof_read, buf, SIMPLE_ARCHIVER_BUFFER_SIZE);
if (read_ret < 0) {
if (errno == EAGAIN || errno == EWOULDBLOCK) {
// Non-blocking read.
nanosleep(&nonblock_sleep, NULL);
} else {
fprintf(stderr,
"ERROR: Reading from compressor, pipe read error!\n");
@ -2625,9 +2456,7 @@ int simple_archiver_write_v1(FILE *out_f, SDArchiverState *state,
// Write compressed chunk.
while (!feof(temp_fd)) {
if (is_sig_int_occurred) {
return SDAS_SIGINT;
} else if (ferror(temp_fd)) {
if (ferror(temp_fd)) {
return SDAS_INTERNAL_ERROR;
}
size_t fread_ret = fread(buf, 1, SIMPLE_ARCHIVER_BUFFER_SIZE, temp_fd);
@ -2660,9 +2489,6 @@ int simple_archiver_write_v1(FILE *out_f, SDArchiverState *state,
fwrite(non_c_chunk_size, 8, 1, out_f);
for (uint64_t file_idx = 0; file_idx < *((uint64_t *)chunk_c_node->data);
++file_idx) {
if (is_sig_int_occurred) {
return SDAS_SIGINT;
}
file_node = file_node->next;
if (file_node == files_list->tail) {
return SDAS_INTERNAL_ERROR;
@ -2695,12 +2521,6 @@ int simple_archiver_write_v1(FILE *out_f, SDArchiverState *state,
int simple_archiver_parse_archive_info(FILE *in_f, int_fast8_t do_extract,
const SDArchiverState *state) {
#if SIMPLE_ARCHIVER_PLATFORM == SIMPLE_ARCHIVER_PLATFORM_COSMOPOLITAN || \
SIMPLE_ARCHIVER_PLATFORM == SIMPLE_ARCHIVER_PLATFORM_MAC || \
SIMPLE_ARCHIVER_PLATFORM == SIMPLE_ARCHIVER_PLATFORM_LINUX
signal(SIGINT, handle_sig_int);
#endif
uint8_t buf[32];
memset(buf, 0, 32);
uint16_t u16;
@ -2817,10 +2637,6 @@ int simple_archiver_parse_archive_version_0(FILE *in_f, int_fast8_t do_extract,
simple_archiver_helper_32_bit_be(&u32);
fprintf(stderr, "File count is %u\n", u32);
if (is_sig_int_occurred) {
return SDAS_SIGINT;
}
const uint32_t size = u32;
const size_t digits = simple_archiver_helper_num_digits(size);
char format_str[128];
@ -2843,9 +2659,6 @@ int simple_archiver_parse_archive_version_0(FILE *in_f, int_fast8_t do_extract,
}
}
for (uint32_t idx = 0; idx < size; ++idx) {
if (is_sig_int_occurred) {
return SDAS_SIGINT;
}
skip = 0;
fprintf(stderr, format_str, idx + 1, size);
if (feof(in_f) || ferror(in_f)) {
@ -2858,9 +2671,6 @@ int simple_archiver_parse_archive_version_0(FILE *in_f, int_fast8_t do_extract,
simple_archiver_helper_cleanup_malloced))) void *out_f_name = NULL;
__attribute__((cleanup(simple_archiver_helper_cleanup_FILE))) FILE *out_f =
NULL;
__attribute__((cleanup(
cleanup_overwrite_filename_delete_simple))) char *to_overwrite_dest =
NULL;
if (u16 < SIMPLE_ARCHIVER_BUFFER_SIZE) {
if (fread(buf, 1, u16 + 1, in_f) != (size_t)u16 + 1) {
return SDAS_INVALID_FILE;
@ -2890,14 +2700,12 @@ int simple_archiver_parse_archive_version_0(FILE *in_f, int_fast8_t do_extract,
if (fd == -1) {
if (errno == ELOOP) {
// Is an existing symbolic file.
// Defer deletion to after "is invalid" check.
to_overwrite_dest = strdup((const char *)buf);
unlink((const char *)buf);
}
} else {
close(fd);
// Is an existing file.
// Defer deletion to after "is invalid" check.
to_overwrite_dest = strdup((const char *)buf);
unlink((const char *)buf);
}
}
if (!skip) {
@ -2939,14 +2747,12 @@ int simple_archiver_parse_archive_version_0(FILE *in_f, int_fast8_t do_extract,
if (fd == -1) {
if (errno == ELOOP) {
// Is an existing symbolic file.
// Defer deletion to after "is invalid" check.
to_overwrite_dest = strdup((const char *)uc_heap_buf);
unlink((const char *)uc_heap_buf);
}
} else {
close(fd);
// Is an existing file.
// Defer deletion to after "is invalid" check.
to_overwrite_dest = strdup((const char *)uc_heap_buf);
unlink((const char *)uc_heap_buf);
}
}
if (!skip) {
@ -2961,17 +2767,6 @@ int simple_archiver_parse_archive_version_0(FILE *in_f, int_fast8_t do_extract,
return SDAS_INVALID_FILE;
}
// Check for "invalid entry" flag.
if ((buf[1] & 0x8) != 0) {
free(to_overwrite_dest);
to_overwrite_dest = NULL;
fprintf(stderr, " This file entry was marked invalid, skipping...\n");
continue;
} else {
// Do deferred overwrite action: remove existing file/symlink.
cleanup_overwrite_filename_delete_simple(&to_overwrite_dest);
}
#if SIMPLE_ARCHIVER_PLATFORM == SIMPLE_ARCHIVER_PLATFORM_COSMOPOLITAN || \
SIMPLE_ARCHIVER_PLATFORM == SIMPLE_ARCHIVER_PLATFORM_MAC || \
SIMPLE_ARCHIVER_PLATFORM == SIMPLE_ARCHIVER_PLATFORM_LINUX
@ -3193,21 +2988,11 @@ int simple_archiver_parse_archive_version_0(FILE *in_f, int_fast8_t do_extract,
char recv_buf[SIMPLE_ARCHIVER_BUFFER_SIZE];
size_t amount_to_read;
while (!write_pipe_done || !read_pipe_done) {
if (is_sig_int_occurred) {
if (pipe_into_cmd[1] >= 0) {
close(pipe_into_cmd[1]);
pipe_into_cmd[1] = -1;
}
if (pipe_outof_cmd[0] >= 0) {
close(pipe_outof_cmd[0]);
pipe_outof_cmd[0] = -1;
}
return SDAS_SIGINT;
} else if (is_sig_pipe_occurred) {
if (is_sig_pipe_occurred) {
fprintf(stderr,
"WARNING: Failed to write to decompressor (SIGPIPE)! "
"Invalid decompressor cmd?\n");
return SDAS_INTERNAL_ERROR;
return 1;
}
// Read from file.
@ -3232,12 +3017,10 @@ int simple_archiver_parse_archive_version_0(FILE *in_f, int_fast8_t do_extract,
write_again = 0;
if (compressed_file_size == 0) {
close(pipe_into_cmd[1]);
pipe_into_cmd[1] = -1;
write_pipe_done = 1;
}
} else if (write_ret == -1) {
if (errno == EAGAIN || errno == EWOULDBLOCK) {
nanosleep(&nonblock_sleep, NULL);
write_again = 1;
} else {
// Error.
@ -3281,7 +3064,6 @@ int simple_archiver_parse_archive_version_0(FILE *in_f, int_fast8_t do_extract,
} else if (read_ret == -1) {
if (errno == EAGAIN || errno == EWOULDBLOCK) {
// No bytes to read yet.
nanosleep(&nonblock_sleep, NULL);
} else {
// Error.
fprintf(stderr,
@ -3293,7 +3075,6 @@ int simple_archiver_parse_archive_version_0(FILE *in_f, int_fast8_t do_extract,
// EOF.
read_pipe_done = 1;
close(pipe_outof_cmd[0]);
pipe_outof_cmd[0] = -1;
simple_archiver_helper_cleanup_FILE(&out_f);
} else {
// Invalid state (unreachable?), error.
@ -3593,10 +3374,6 @@ int simple_archiver_parse_archive_version_0(FILE *in_f, int_fast8_t do_extract,
}
}
if (is_sig_int_occurred) {
return SDAS_SIGINT;
}
return SDAS_SUCCESS;
}
@ -3681,10 +3458,6 @@ int simple_archiver_parse_archive_version_1(FILE *in_f, int_fast8_t do_extract,
}
}
if (is_sig_int_occurred) {
return SDAS_SIGINT;
}
// Link count.
if (fread(buf, 1, 4, in_f) != 4) {
return SDAS_INVALID_FILE;
@ -3693,21 +3466,11 @@ int simple_archiver_parse_archive_version_1(FILE *in_f, int_fast8_t do_extract,
simple_archiver_helper_32_bit_be(&u32);
for (uint32_t idx = 0; idx < u32; ++idx) {
if (is_sig_int_occurred) {
return SDAS_SIGINT;
}
fprintf(stderr, "SYMLINK %3u of %3u\n", idx + 1, u32);
if (fread(buf, 1, 2, in_f) != 2) {
return SDAS_INVALID_FILE;
}
const uint_fast8_t absolute_preferred = (buf[0] & 1) ? 1 : 0;
const uint_fast8_t is_invalid = (buf[1] & 4) ? 1 : 0;
if (is_invalid) {
fprintf(stderr,
" WARNING: This symlink entry was marked invalid (not a safe "
"link)!\n");
}
#if SIMPLE_ARCHIVER_PLATFORM == SIMPLE_ARCHIVER_PLATFORM_COSMOPOLITAN || \
SIMPLE_ARCHIVER_PLATFORM == SIMPLE_ARCHIVER_PLATFORM_MAC || \
@ -3717,7 +3480,7 @@ int simple_archiver_parse_archive_version_1(FILE *in_f, int_fast8_t do_extract,
uint_fast8_t link_extracted = 0;
uint_fast8_t skip_due_to_map = 0;
uint_fast8_t skip_due_to_invalid = is_invalid ? 1 : 0;
uint_fast8_t skip_due_to_invalid = 0;
if (fread(buf, 1, 2, in_f) != 2) {
return SDAS_INVALID_FILE;
@ -3922,9 +3685,6 @@ int simple_archiver_parse_archive_version_1(FILE *in_f, int_fast8_t do_extract,
const uint32_t chunk_count = u32;
for (uint32_t chunk_idx = 0; chunk_idx < chunk_count; ++chunk_idx) {
if (is_sig_int_occurred) {
return SDAS_SIGINT;
}
fprintf(stderr, "CHUNK %3u of %3u\n", chunk_idx + 1, chunk_count);
if (fread(buf, 1, 4, in_f) != 4) {
@ -4130,9 +3890,6 @@ int simple_archiver_parse_archive_version_1(FILE *in_f, int_fast8_t do_extract,
ssize_t has_hold = -1;
while (node->next != file_info_list->tail) {
if (is_sig_int_occurred) {
return SDAS_SIGINT;
}
node = node->next;
const SDArchiverInternalFileInfo *file_info = node->data;
fprintf(stderr, " FILE %3u of %3u: %s\n", ++file_idx, file_count,
@ -4164,14 +3921,11 @@ int simple_archiver_parse_archive_version_1(FILE *in_f, int_fast8_t do_extract,
fprintf(stderr,
" WARNING: File already exists and "
"\"--overwrite-extract\" is not specified, skipping!\n");
int ret = read_decomp_to_out_file(
NULL, pipe_outof_read, (char *)buf,
SIMPLE_ARCHIVER_BUFFER_SIZE, file_info->file_size,
&pipe_into_write, &chunk_remaining, in_f, hold_buf,
&has_hold);
if (ret != SDAS_SUCCESS) {
return ret;
}
read_decomp_to_out_file(NULL, pipe_outof_read, (char *)buf,
SIMPLE_ARCHIVER_BUFFER_SIZE,
file_info->file_size, &pipe_into_write,
&chunk_remaining, in_f, hold_buf,
&has_hold);
continue;
}
}
@ -4235,13 +3989,10 @@ int simple_archiver_parse_archive_version_1(FILE *in_f, int_fast8_t do_extract,
}
} else {
#else
// } (This comment exists so that vim can correctly match curly-braces).
// } (This comment exists so that vim can correctly match curly-braces.
if (!is_compressed) {
#endif
while (node->next != file_info_list->tail) {
if (is_sig_int_occurred) {
return SDAS_SIGINT;
}
node = node->next;
const SDArchiverInternalFileInfo *file_info = node->data;
fprintf(stderr, " FILE %3u of %3u: %s\n", ++file_idx, file_count,

3
src/archiver.h
View file

@ -52,8 +52,7 @@ typedef enum SDArchiverStateReturns {
SDAS_FAILED_TO_CREATE_MAP,
SDAS_FAILED_TO_EXTRACT_SYMLINK,
SDAS_FAILED_TO_CHANGE_CWD,
SDAS_INVALID_WRITE_VERSION,
SDAS_SIGINT
SDAS_INVALID_WRITE_VERSION
} SDArchiverStateReturns;
/// Returned pointer must not be freed.

15
src/parser.c
View file

@ -168,9 +168,6 @@ void simple_archiver_print_usage(void) {
fprintf(stderr, "--overwrite-extract : allows overwriting when extracting\n");
fprintf(stderr,
"--no-abs-symlink : do not store absolute paths for symlinks\n");
fprintf(stderr,
"--no-safe-links : keep symlinks that link to outside archive "
"contents\n");
fprintf(stderr,
"--temp-files-dir <dir> : where to store temporary files created "
"when compressing (defaults to current working directory)\n");
@ -180,9 +177,6 @@ void simple_archiver_print_usage(void) {
fprintf(stderr,
"--chunk-min-size <bytes> : v1 file format minimum chunk size "
"(default 4194304 or 4MiB)\n");
fprintf(stderr,
"--no-pre-sort-files : do NOT pre-sort files by size (by default "
"enabled so that the first file is the largest)\n");
fprintf(stderr,
"-- : specifies remaining arguments are files to archive/extract\n");
fprintf(
@ -196,7 +190,7 @@ void simple_archiver_print_usage(void) {
SDArchiverParsed simple_archiver_create_parsed(void) {
SDArchiverParsed parsed;
parsed.flags = 0x40;
parsed.flags = 0;
parsed.filename = NULL;
parsed.compressor = NULL;
parsed.decompressor = NULL;
@ -306,11 +300,6 @@ int simple_archiver_parse_args(int argc, const char **argv,
out->flags |= 0x8;
} else if (strcmp(argv[0], "--no-abs-symlink") == 0) {
out->flags |= 0x20;
} else if (strcmp(argv[0], "--no-safe-links") == 0) {
out->flags |= 0x80;
fprintf(stderr,
"NOTICE: Disabling safe-links, symlinks that point to outside "
"archived files will be preserved!\n");
} else if (strcmp(argv[0], "--temp-files-dir") == 0) {
if (argc < 2) {
fprintf(stderr, "ERROR: --temp-files-dir is missing an argument!\n");
@ -355,8 +344,6 @@ int simple_archiver_parse_args(int argc, const char **argv,
}
--argc;
++argv;
} else if (strcmp(argv[0], "--no-pre-sort-files") == 0) {
out->flags &= 0xFFFFFFBF;
} else if (argv[0][0] == '-' && argv[0][1] == '-' && argv[0][2] == 0) {
is_remaining_args = 1;
} else if (argv[0][0] != '-') {

2
src/parser.h
View file

@ -35,8 +35,6 @@ typedef struct SDArchiverParsed {
/// 0b xxxx 1xxx - Allow extract overwrite.
/// 0b xxx1 xxxx - Create archive to stdout or read archive from stdin.
/// 0b xx1x xxxx - Do not save absolute paths for symlinks.
/// 0b x1xx xxxx - Sort files by size before archiving.
/// 0b 1xxx xxxx - No safe links.
uint32_t flags;
/// Null-terminated string.
char *filename;

4
src/test.c
View file

@ -109,7 +109,7 @@ int main(void) {
CHECK_TRUE(strcmp("doop", parsed.working_files[1]) == 0);
CHECK_TRUE(parsed.working_files[2] == NULL);
CHECK_TRUE(parsed.filename == NULL);
CHECK_TRUE(parsed.flags == 0x40);
CHECK_TRUE(parsed.flags == 0);
simple_archiver_free_parsed(&parsed);
@ -126,7 +126,7 @@ int main(void) {
CHECK_TRUE(strcmp("../../.prev_dir_file", parsed.working_files[2]) == 0);
CHECK_TRUE(parsed.working_files[3] == NULL);
CHECK_TRUE(strcmp("the_filename", parsed.filename) == 0);
CHECK_TRUE(parsed.flags == 0x41);
CHECK_TRUE(parsed.flags == 1);
simple_archiver_free_parsed(&parsed);
}