Quantcast
Channel: Admins Goodies » compression
Viewing all articles
Browse latest Browse all 10

How can i handle huge log files?

$
0
0

Question

I am running SSD servers because I have great experiences with performance.
However I have webservers which serve lots of requests and the uncompresssed logfiles of one day tend to grow out the database.

I have usual standard format Webserver log files.

Is there maby a way to compress them “on the fly”?

Answer

Logrotate

This utility can automatically inform your daemon to switch logs as it renames and compresses them. Since daily is not enough, you could also run it hourly. Even if logged hourly, you can still concatenate them by day after compression with GZip.

Compressed filesystem

Btrfs can be setup with transparent encryption, automatically saving space in the background. Some other filesystems (ZFS, etc.) can also accomplish this.

Compression pipe

Apache will log directly to a pipe and most applications will write to a name pipe (mkfifo) which you can attach to your compression program.

Answered by Jeff Ferland

Viewing all articles
Browse latest Browse all 10

Trending Articles