Truncating a file while it's being used (Linux)

Redirect the output using >> instead of >. This will allow you to truncate the file without the file growing back to its original size. Also, don't forget to redirect STDERR (2>&1).\n\nSo the end result would be: myprogram >> myprogram.log 2>&1 &",

Try > file.\n\n\n\nUpdate regarding the comments: it works nicely for me:\n\nrobert@rm:~> echo \"content\" > test-file\nrobert@rm:~> cat test-file \ncontent\nrobert@rm:~> > test-file\nrobert@rm:~> cat test-file \n",

I had a similar issue on redhat v6, echo > file or > file was causing apache and tomcat to go faulty as log files would become inaccessible to them.\n\nAnd the fix was strange\n\necho \" \" > file\n\n\nwould clean the file and not cause any problem.",

In Linux (actually all unicies) files are created when they are opened and deleted when nothing holds a reference to them. In this case the program that opened it and the directory it was opened 'in' hold references to the file. When the cp program wants to write to the file it gets a reference to it from the directory, writes a length of zero into the metadata stored in the directory (this is a slight simplification) and gives up the handle. Then the original program, still holding the original file handle, writes some more data to the file and saves what it thinks the length should be. \neven if you where to delete the file from the directory the program would continue to write data to it (and use up disc space) even though no other program would have any way of referencing it. \nin short once the program has a reference (handle) to a file nothing you do is going to change that. \nthere are in theory ways of modifying the programs behavior by setting LD_LIBRARY_PATH to include a program that intercepts all the file access system calls. I recall seeing something like this somewhere though cant recall the name. ",

Take a look at the utility split(1), part of GNU Coreutils.",

as the file is being used, if you try to nullify it or something like that, sometimes it might \"confuse\" the app that's writing into the log file and it might not log anything after that. \n\nWhat I'd try ot do is to set up a kind of proxy/filter for that log, instead of redirecting to file, redirect to a process or something that would get input and write to a rolling file.\n\nMaybe it can be done by script otherwise you could write a simple app for that ( java or something else ). The impact on app performance should be quite small, but you'll have to run some tests.\n\nBtw, your app, is it a stand-alone, web app, ... ? Maybe there are other options to be investigated.\n\nEdit: there's also an Append Redirection Operator >> that I've personally never used, but it might not lock the file.",

I downloaded and compiled the latest coreutils so I could have truncate available.\n\nRan ./configure and make, but did not run make install.\n\nAll the compiled utilities appear in the \"src\" folder.\n\nI ran \n\n[path]/src/truncate -s 1024000 textfileineedtotruncate.log \n\non a 1.7 GB log file.\n\nIt did not change the size listed when using ls -l, but it did free up all the disk space - which is what I really needed to do before /var filled up and killed the process.\n\nThanks for the tip on \"truncate\"!",

Did you check the behavior of any signals like SIGHUP to the third party product, to see if it will start logging a fresh file? You would move the old file to a permanent name, first.\n\nkill -HUP [process-id]\n\nAnd then it would start writing out again.\n\nAlternatively (as Billy suggested) maybe redirecting the output from the application to a logging program like multilog or the one that is commonly used with Apache, known as cronolog. Then you'll have more fine grained control of where everything goes before it is written to that initial file descriptor (file), which is really all it is.",

As of coreutils 7.0, there is a truncate command.",

instead of redirecting it to a file you could pipe it to a program that automatically rotates the file by closing it, moving it and opening a new one every time it gets too big. ",

The interesting thing about those regrown files is that the first 128 KB or so will be all zeroes after you truncate the file by copying /dev/null over it. This happens because the file is truncated to zero length, but the file descriptor in the application still points immediately after its last write. When it writes again, the file system treats the start of the file as all zero bytes - without actually writing the zeroes to disk.\n\nIdeally, you should ask the vendor of the application to open the log file with the O_APPEND flag. This means that after you truncate the file, the next write will implicitly seek to the end of the file (meaning back to offset zero) and then write the new information.\n\n\n\nThis code rigs standard output so it is in O_APPEND mode and then invokes the command given by its arguments (rather like nice runs a command after adjusting its nice-level, or nohup runs a command after fixing things so it ignores SIGHUP).\n\n#include <stdlib.h>\n#include <unistd.h>\n#include <fcntl.h>\n#include <stdio.h>\n#include <stdarg.h>\n#include <string.h>\n#include <errno.h>\n\nstatic char *arg0 = \"<unknown>\";\n\nstatic void error(const char *fmt, ...)\n{\n va_list args;\n int errnum = errno;\n fprintf(stderr, \"%s: \", arg0);\n va_start(args, fmt);\n vfprintf(stderr, fmt, args);\n va_end(args);\n if (errnum != 0)\n fprintf(stderr, \" (%d: %s)\", errnum, strerror(errnum));\n putc('\\n', stderr);\n fflush(0);\n exit(1);\n}\n\nint main(int argc, char **argv)\n{\n int attr;\n arg0 = argv[0];\n\n if (argc < 2)\n error(\"Usage: %s cmd [arg ...]\", arg0);\n if ((attr = fcntl(1, F_GETFL, &attr)) < 0)\n error(\"fcntl(F_GETFL) failed\");\n attr |= O_APPEND;\n if (fcntl(1, F_SETFL, attr) != 0)\n error(\"fcntl(F_SETFL) failed\");\n execvp(argv[1], &argv[1]);\n error(\"failed to exec %s\", argv[1]);\n return(1);\n}\n\n\nMy testing of it was somewhat casual, but just barely enough to persuade me that it worked.\n\n\n\nSimpler alternative\n\nBilly notes in his answer that '>>' is the append operator - and indeed, on Solaris 10, bash (version 3.00.16(1)) does use the O_APPEND flag - thereby making the code above unnecessary, as shown ('Black JL:' is my prompt on this machine):\n\nBlack JL: truss -o bash.truss bash -c \"echo Hi >> x3.29\"\nBlack JL: grep open bash.truss\nopen(\"/var/ld/ld.config\", O_RDONLY) Err#2 ENOENT\nopen(\"/usr/lib/libcurses.so.1\", O_RDONLY) = 3\nopen(\"/usr/lib/libsocket.so.1\", O_RDONLY) = 3\nopen(\"/usr/lib/libnsl.so.1\", O_RDONLY) = 3\nopen(\"/usr/lib/libdl.so.1\", O_RDONLY) = 3\nopen(\"/usr/lib/libc.so.1\", O_RDONLY) = 3\nopen(\"/platform/SUNW,Ultra-4/lib/libc_psr.so.1\", O_RDONLY) = 3\nopen64(\"/dev/tty\", O_RDWR|O_NONBLOCK) = 3\nstat64(\"/usr/openssl/v0.9.8e/bin/bash\", 0xFFBFF2A8) Err#2 ENOENT\nopen64(\"x3.29\", O_WRONLY|O_APPEND|O_CREAT, 0666) = 3\nBlack JL:\n\n\nUse append redirection rather than the wrapper ('cantrip') code above. This just goes to show that when you use one particular technique for other (valid) purposes, adapting it to yet another is not necessarily the simplest mechanism - even though it works.",

@Hobo use freopen(), it reuses stream to either open the file specified by filename or to change its access mode.\nIf a new filename is specified, the function first attempts to close any file already associated with stream (third parameter) and disassociates it. Then, independently of whether that stream was successfully closed or not, freopen opens the file specified by filename and associates it with the stream just as fopen would do using the specified mode.\n\nif a thirdparty binary is generating logs we need to write a wrapper which will rotate the logs, and thirdparty will run in proxyrun thread as below.\n\n#include <pthread.h>\n#include <stdio.h>\n#include <unistd.h>\n#include <unistd.h>\n#include <string.h>\n\nusing namespace std;\n\nextern \"C\" void * proxyrun(void * pArg){\n static int lsiLineNum = 0;\n while(1) \n {\n printf(\"\\nLOGGER: %d\",++lsiLineNum);\n fflush(stdout);\n }\n return NULL;\n}\n\n\nint main(int argc, char **argv)\n{\n pthread_t lThdId;\n if(0 != pthread_create(&lThdId, NULL, proxyrun, NULL))\n {\n return 1;\n }\n\n char lpcFileName[256] = {0,};\n\n static int x = 0;\n\n while(1)\n {\n printf(\"\\n<<<MAIN SLEEP>>>\");\n fflush(stdout);\n sprintf(lpcFileName, \"/home/yogesh/C++TestPrograms/std.txt%d\",++x);\n freopen(lpcFileName,\"w\",stdout);\n sleep(10);\n }\n\n return 0;\n}\n",