Subject: Use Ramdisk for Caching
(I have been working on an issue related to squid slowing down gradually thus slowing down internet access by network users. As it made me more enthusiastic what could be the reason or could be there an alternative way to resolve this. Thus, I can across the so called RAMDISK and tried to use it with squid.
I tried my way to use RAMDISK with SQUID. I found squid was doing good....so here I blogged......to get the better view of my result.
INTRODUCTION:
RAM disk is a section of the RAM (memory) which is configured as a disk drive. Data stored in RAM disk can be accessed faster than the data stored in a hard disk drive.
Usually, Squid is configured to cache its information of visited websites in the hard disk. Using a RAM Disk for storing this information will enable the squid to access the cache more faster than the one stored in cache. Though, data stored in the RAM DISK are TEMPORARY (i.e. data will be stored only until the system is up and running), I recommend to have multiple cache directories storage: one in RAMDISK and another as well as in Hard-disk.
I. CREATING RAMDISK for SQUID cache storage:
1) Check if ramdisk has been created?
By default only ram0 - ram15 are usable
root@lino90:~# ls -l /dev/ram*
lrwxrwxrwx 1 root root 4 Jul 20 12:37 /dev/ram -> ram1
brw-r----- 1 root disk 1, 0 Jul 20 12:38 /dev/ram0
brw-r----- 1 root disk 1, 1 Jul 20 12:36 /dev/ram1
brw-r----- 1 root disk 1, 10 Jul 20 12:36 /dev/ram10
brw-r----- 1 root disk 1, 11 Jul 20 12:36 /dev/ram11
brw-r----- 1 root disk 1, 12 Jul 20 12:36 /dev/ram12
brw-r----- 1 root disk 1, 13 Jul 20 12:36 /dev/ram13
brw-r----- 1 root disk 1, 14 Jul 20 12:36 /dev/ram14
brw-r----- 1 root disk 1, 15 Jul 20 12:36 /dev/ram15
brw-r----- 1 root disk 1, 2 Jul 20 12:36 /dev/ram2
brw-r----- 1 root disk 1, 3 Jul 20 12:36 /dev/ram3
brw-r----- 1 root disk 1, 4 Jul 20 12:36 /dev/ram4
brw-r----- 1 root disk 1, 5 Jul 20 12:36 /dev/ram5
brw-r----- 1 root disk 1, 6 Jul 20 12:36 /dev/ram6
brw-r----- 1 root disk 1, 7 Jul 20 12:36 /dev/ram7
brw-r----- 1 root disk 1, 8 Jul 20 12:36 /dev/ram8
brw-r----- 1 root disk 1, 9 Jul 20 12:36 /dev/ram9
lrwxrwxrwx 1 root root 4 Jul 20 12:37 /dev/ramdisk -> ram0
2) To Check the size of ramdisk:
root@lino90:~# dmesg | grep RAMDISK
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
root@lino90:~#
This shows that the default ramdisk size is 4MB.
3) To increase the size of the ramdisk for eg. 16MB
A command has to be passed to the kernel during booting. This can be done in the grub.conf by passing an entry in ramdisk_size=[enter the size in 1024-byte blocks]
root@lino90:~# vi /etc/grub.conf
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE: You do not have a /boot partition. This means that
# all kernel and initrd paths are relative to /, eg.
# root (hd0,0)
# kernel /boot/vmlinuz-version ro root=/dev/sda1
# initrd /boot/initrd-version.img
# boot=/dev/sda
default=0
timeout=5
splashimage=(hd0,0)/boot/grub/splash.xpm.gz
hiddenmenu
title Red Hat Enterprise Linux Server (2.6.18-8.el5)
root (hd0,0)
kernel /boot/vmlinuz-2.6.18-8.el5 ro root=LABEL=/1 rhgb quiet ramdisk_size=16000
initrd /boot/initrd-2.6.18-8.el5.img
Save the file and reboot. After the reboot, confirm with the below command to check the new size of the ramdisk
root@lino90:~# dmesg | grep RAMDISK
RAMDISK driver initialized: 16 RAM disks of 16000K size 1024 blocksize
root@lino90:~#
4) Format the ramdisk as a journaling file system eg.ext2 file system. In case if only one ramdisk, we will use /dev/ram0
root@lino90:~# mke2fs -m 0 /dev/ram0
mke2fs 1.41.3 (12-Oct-2008)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
16384 inodes, 65536 blocks
0 blocks (0.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=67108864
8 block groups
8192 blocks per group, 8192 fragments per group
2048 inodes per group
Superblock backups stored on blocks:
8193, 24577, 40961, 57345
Writing inode tables: done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 29 mounts or
80 days, whichever comes first. Use tune2fs -c or -i to override.
5) mounting ramdisk
a) Create a directory (In case of squid you can create the directory where the squid cache is stored)
root@lino90:~# mkdir /var/spool/squid_ram
b) mount the ramdisk to the above directory
root@lino90:~ # mount /dev/ram0 /var/spool/squid_ram
c) To verify the ramdisk mount
root@lino90:~# mount | grep ram0
OR
root@lino90:~# df -h | grep ram0
d) Change the ownership as per squid requirement
root@lino90:~# chown squid:squid -R /var/spool/squid_mem
e) Change the permission as per squid requirement
root@lino90:~# chmod 755 -R /var/spool/squid_mem
f) To check the details of the new ramdisk
root@lino90:~# tune2fs -l /dev/ram0
g) To create and Automount ramdisk at boot time add the below entry in rc.local
root@lino90:~# vi /etc/rc.local
/sbin/mke2fs -q -m 0 /dev/ram0
/bin/mount /dev/ram0 /var/spool/squid_mem
/bin/chown squid:squid /var/spool/squid_mem
/bin/chmod 0755 /var/spool/squid_mem
/usr/sbin/squid -z
Save the rc.local and exit.
II. CONFIGURING SQUID TO USE RAMDISK
1) Adding Ramdisk for cache store in addition to the existing cache store.
root@lino90:~# vi /etc/squid/squid.conf
2) Search for cache_dir entry in your squid.conf. Your orginal entry could be like below:
#
#Default:
cache_dir aufs /var/spool/squid 1000 20 256
Add the RAMDISK mount directory in the squid.conf
#
#Default:
cache_dir aufs /var/spool/squid_mem 45 100 100
cache_dir aufs /var/spool/squid 1000 20 256
The setting of "cache_dir aufs /var/spool/squid_mem 45 100 100" is ideally for a 50 MB of ramdisk. In this case we have configured 45MB out of 50MB randisk for cache store, leaving 5MB unused. The number of irst level sub-directories created will be 100 and number of second level sub-directories created will be 100
Here, you need to check if squid can create the defined number of 1st level and 2nd level of cache directoies in the RAMDISK by running the below command
root@lino90:~# squid -z
In case if it is unable to create it will display an error
root@lino90:~# squid -z
Creating Swap Directories
FATAL: Failed to make swap directory /var/spool/squid_mem/77/4B: (28) No space left on device
Squid Cache (Version 2.6.STABLE6): Terminated abnormally.
CPU Usage: 0.157 seconds = 0.000 user + 0.157 sys
Maximum Resident Size: 0 KB
Page faults with physical i/o: 0
Then, in this case you will have to modify the number of cache directories (a balance number of cache directories)
Run the command squid -z, until you only get the below message
root@lino90:~# squid -z
2010/07/21 14:01:17
Creating Swap Directories
root@lino90:~#
Once Done start the squid service.
root@lino90:~ /etc/init.d/squid start
Note: Disable the squid service startup during bootup, as this will fail because at that time ramdisk will not be mounted.
root@lino90:~ chkconfig squid off
End Note: I scheduled a regular backup of the /var/spool/squid_mem, so the same can be restored in case if the system restarts. This, I have configured to load on booting.
(I have been working on an issue related to squid slowing down gradually thus slowing down internet access by network users. As it made me more enthusiastic what could be the reason or could be there an alternative way to resolve this. Thus, I can across the so called RAMDISK and tried to use it with squid.
I tried my way to use RAMDISK with SQUID. I found squid was doing good....so here I blogged......to get the better view of my result.
INTRODUCTION:
RAM disk is a section of the RAM (memory) which is configured as a disk drive. Data stored in RAM disk can be accessed faster than the data stored in a hard disk drive.
Usually, Squid is configured to cache its information of visited websites in the hard disk. Using a RAM Disk for storing this information will enable the squid to access the cache more faster than the one stored in cache. Though, data stored in the RAM DISK are TEMPORARY (i.e. data will be stored only until the system is up and running), I recommend to have multiple cache directories storage: one in RAMDISK and another as well as in Hard-disk.
I. CREATING RAMDISK for SQUID cache storage:
1) Check if ramdisk has been created?
root@lino90:~# ls -l /dev/ram*
By default only ram0 - ram15 are usable
root@lino90:~# ls -l /dev/ram*
lrwxrwxrwx 1 root root 4 Jul 20 12:37 /dev/ram -> ram1
brw-r----- 1 root disk 1, 0 Jul 20 12:38 /dev/ram0
brw-r----- 1 root disk 1, 1 Jul 20 12:36 /dev/ram1
brw-r----- 1 root disk 1, 10 Jul 20 12:36 /dev/ram10
brw-r----- 1 root disk 1, 11 Jul 20 12:36 /dev/ram11
brw-r----- 1 root disk 1, 12 Jul 20 12:36 /dev/ram12
brw-r----- 1 root disk 1, 13 Jul 20 12:36 /dev/ram13
brw-r----- 1 root disk 1, 14 Jul 20 12:36 /dev/ram14
brw-r----- 1 root disk 1, 15 Jul 20 12:36 /dev/ram15
brw-r----- 1 root disk 1, 2 Jul 20 12:36 /dev/ram2
brw-r----- 1 root disk 1, 3 Jul 20 12:36 /dev/ram3
brw-r----- 1 root disk 1, 4 Jul 20 12:36 /dev/ram4
brw-r----- 1 root disk 1, 5 Jul 20 12:36 /dev/ram5
brw-r----- 1 root disk 1, 6 Jul 20 12:36 /dev/ram6
brw-r----- 1 root disk 1, 7 Jul 20 12:36 /dev/ram7
brw-r----- 1 root disk 1, 8 Jul 20 12:36 /dev/ram8
brw-r----- 1 root disk 1, 9 Jul 20 12:36 /dev/ram9
lrwxrwxrwx 1 root root 4 Jul 20 12:37 /dev/ramdisk -> ram0
2) To Check the size of ramdisk:
root@lino90:~# dmesg | grep RAMDISK
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
root@lino90:~#
This shows that the default ramdisk size is 4MB.
3) To increase the size of the ramdisk for eg. 16MB
A command has to be passed to the kernel during booting. This can be done in the grub.conf by passing an entry in ramdisk_size=[enter the size in 1024-byte blocks]
root@lino90:~# vi /etc/grub.conf
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE: You do not have a /boot partition. This means that
# all kernel and initrd paths are relative to /, eg.
# root (hd0,0)
# kernel /boot/vmlinuz-version ro root=/dev/sda1
# initrd /boot/initrd-version.img
# boot=/dev/sda
default=0
timeout=5
splashimage=(hd0,0)/boot/grub/splash.xpm.gz
hiddenmenu
title Red Hat Enterprise Linux Server (2.6.18-8.el5)
root (hd0,0)
kernel /boot/vmlinuz-2.6.18-8.el5 ro root=LABEL=/1 rhgb quiet ramdisk_size=16000
initrd /boot/initrd-2.6.18-8.el5.img
Save the file and reboot. After the reboot, confirm with the below command to check the new size of the ramdisk
root@lino90:~# dmesg | grep RAMDISK
RAMDISK driver initialized: 16 RAM disks of 16000K size 1024 blocksize
root@lino90:~#
4) Format the ramdisk as a journaling file system eg.ext2 file system. In case if only one ramdisk, we will use /dev/ram0
root@lino90:~# mke2fs -m 0 /dev/ram0
mke2fs 1.41.3 (12-Oct-2008)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
16384 inodes, 65536 blocks
0 blocks (0.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=67108864
8 block groups
8192 blocks per group, 8192 fragments per group
2048 inodes per group
Superblock backups stored on blocks:
8193, 24577, 40961, 57345
Writing inode tables: done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 29 mounts or
80 days, whichever comes first. Use tune2fs -c or -i to override.
5) mounting ramdisk
a) Create a directory (In case of squid you can create the directory where the squid cache is stored)
root@lino90:~# mkdir /var/spool/squid_ram
b) mount the ramdisk to the above directory
root@lino90:~ # mount /dev/ram0 /var/spool/squid_ram
c) To verify the ramdisk mount
root@lino90:~# mount | grep ram0
OR
root@lino90:~# df -h | grep ram0
d) Change the ownership as per squid requirement
root@lino90:~# chown squid:squid -R /var/spool/squid_mem
e) Change the permission as per squid requirement
root@lino90:~# chmod 755 -R /var/spool/squid_mem
f) To check the details of the new ramdisk
root@lino90:~# tune2fs -l /dev/ram0
g) To create and Automount ramdisk at boot time add the below entry in rc.local
root@lino90:~# vi /etc/rc.local
/sbin/mke2fs -q -m 0 /dev/ram0
/bin/mount /dev/ram0 /var/spool/squid_mem
/bin/chown squid:squid /var/spool/squid_mem
/bin/chmod 0755 /var/spool/squid_mem
/usr/sbin/squid -z
Save the rc.local and exit.
II. CONFIGURING SQUID TO USE RAMDISK
1) Adding Ramdisk for cache store in addition to the existing cache store.
root@lino90:~# vi /etc/squid/squid.conf
2) Search for cache_dir entry in your squid.conf. Your orginal entry could be like below:
#
#Default:
cache_dir aufs /var/spool/squid 1000 20 256
Add the RAMDISK mount directory in the squid.conf
#
#Default:
cache_dir aufs /var/spool/squid_mem 45 100 100
cache_dir aufs /var/spool/squid 1000 20 256
The setting of "cache_dir aufs /var/spool/squid_mem 45 100 100" is ideally for a 50 MB of ramdisk. In this case we have configured 45MB out of 50MB randisk for cache store, leaving 5MB unused. The number of irst level sub-directories created will be 100 and number of second level sub-directories created will be 100
Here, you need to check if squid can create the defined number of 1st level and 2nd level of cache directoies in the RAMDISK by running the below command
root@lino90:~# squid -z
In case if it is unable to create it will display an error
root@lino90:~# squid -z
Creating Swap Directories
FATAL: Failed to make swap directory /var/spool/squid_mem/77/4B: (28) No space left on device
Squid Cache (Version 2.6.STABLE6): Terminated abnormally.
CPU Usage: 0.157 seconds = 0.000 user + 0.157 sys
Maximum Resident Size: 0 KB
Page faults with physical i/o: 0
Then, in this case you will have to modify the number of cache directories (a balance number of cache directories)
Run the command squid -z, until you only get the below message
root@lino90:~# squid -z
2010/07/21 14:01:17
Creating Swap Directories
root@lino90:~#
Once Done start the squid service.
root@lino90:~ /etc/init.d/squid start
Note: Disable the squid service startup during bootup, as this will fail because at that time ramdisk will not be mounted.
root@lino90:~ chkconfig squid off
End Note: I scheduled a regular backup of the /var/spool/squid_mem, so the same can be restored in case if the system restarts. This, I have configured to load on booting.
