– Written by Juan Gabriel Covas 2022
Lsyncd uses a filesystem event interface (inotify or fsevents) to watch for changes to local files and directories. Lsyncd collates these events for several seconds and then spawns one or more processes to synchronize the changes to a remote filesystem.
Motivation: I wanted to compile the shiny, latest version of lsyncd 2.3.1 (2.3.0) instead of the stock version 2.2.2
or 2.2.3
to get some fixes and be able to use delete excluded
from rsync (–delete-excluded
), among other recent fixes.
(Incomplete, but tested on Rocky Linux 9)
Assuming you have Lua 5.4.2 on your EL9 system:
1. Build lua-5.4.2 into /opt/lua-5.4.2…
#!/bin/bash lua_version="5.4.2" gcc --version # compile latest LUA if [[ ! -d lua-$lua_version ]] ;then echo " " echo Compiling latest LUA ${lua_version} ... curl https://www.lua.org/ftp/lua-${lua_version}.tar.gz > lua-${lua_version}.tar.gz tar -zxf lua-${lua_version}.tar.gz cd lua-${lua_version} make INSTALL_TOP=/opt/lua-${lua_version} linux test make INSTALL_TOP=/opt/lua-${lua_version} linux install fi
2. Use the following “cmake” command to build:
cmake .. -DLUA_INCLUDE_DIR=/opt/lua-5.4.2/include -DLUA_LIBRARIES=/usr/lib64/liblua-5.4.so -DLUA_EXECUTABLE=/usr/bin/lua -DLUA_COMPILER=/usr/bin/luac make make install
WooHoo!
Please note that the following instructions are tested under CentOS 7 (soon to be EOL'ed)
1. Install devtoolset-11 modern dev tools
yum install centos-release-scl yum install devtoolset-11
If you want a bash session for devtoolset-11, use:
# scl enable devtoolset-11 bash
If you want use devtoolset-11 inside a bash script, use:
source scl_source enable devtoolset-11
Check gcc version:
# gcc --version gcc (GCC) 11.2.1 20220127 (Red Hat 11.2.1-9)
2. Ensure we have cmake and cmake3
yum install cmake cmake3
2.1 Now the problem is that cmake
is v2.8 and cmake3 the version we need (Error message: CMake 3.10 or higher is required. You are running version 2.8.12.2)
. Thanks to this guide about Installing the latest git/cmake versions on RHEL/Centos.
The relevant part is to execute the following two commands, assuming you have installed cmake AND cmake3:
sudo alternatives --install /usr/local/bin/cmake cmake /usr/bin/cmake 10 \ --slave /usr/local/bin/ctest ctest /usr/bin/ctest \ --slave /usr/local/bin/cpack cpack /usr/bin/cpack \ --slave /usr/local/bin/ccmake ccmake /usr/bin/ccmake \ --family cmake sudo alternatives --install /usr/local/bin/cmake cmake /usr/bin/cmake3 20 \ --slave /usr/local/bin/ctest ctest /usr/bin/ctest3 \ --slave /usr/local/bin/cpack cpack /usr/bin/cpack3 \ --slave /usr/local/bin/ccmake ccmake /usr/bin/ccmake3 \ --family cmake
Then we can set cmake3 as the default “cmake” command:
# alternatives --config cmake There are 2 programs which provide 'cmake'. Selection Command ----------------------------------------------- + 1 cmake (/usr/bin/cmake) * 2 cmake (/usr/bin/cmake3) Enter to keep the current selection[+], or type selection number: 2
Test cmake version:
# cmake --version cmake3 version 3.17.5
3. Install Lua 5.3
Lsyncd requires Lua version >= 5.2 so we're going to try Lua 5.3 from IUS repo for Centos 7
# yum install -y epel-release https://repo.ius.io/ius-release-el7.rpm
Make sure IUS repo is enabled:
# yum-config-manager --enable ius
Install all lua53u packages:
# yum install -y lua53u*
Please note that we want to install ALL packages, therefore lua53u*
←- with a trailing asterisk
Then later we can use the following at cmake ..
line: -DLUA_INCLUDE_DIR=/usr/include/lua-5.3 -DLUA_LIBRARIES=/usr/lib64/liblua-5.3.so -DLUA_EXECUTABLE=/usr/bin/lua-5.3 -DLUA_COMPILER=/usr/bin/luac-5.3
Comment: althought the lsyncd manual says that LUA 5.2 at least is required, the stock version is Lua is 5.1 on CentOS 7. Build of lsyncd (cmake) seems to find OK the 5.1 LUA… so it's strange. For my simple /etc/lscynd.conf
, seems to work, but finally got the way to use at least LUA 5.3 as described above.
4. Download latest version of lsyncd from github:
The 2.3.1 version supports “delete_excluded=true” at rsync
portion of a sync
config, which was important to me.
# wget https://github.com/lsyncd/lsyncd/archive/refs/tags/v2.3.1.tar.gz -O lsyncd-2.3.1.tar.gz # tar xvzf lsyncd-2.3.1.tar.gz # cd lsyncd-2.3.1
5. Build lsyncd
# mkdir build && cd build
If you want to try the stock LUA… which in CentOS 7 is 5.1 (not recommended)
# cmake ..
OR if you installed LUA 5.3 and want to force it, do the following:
Add this line to CMakeLists.txt file, above the line # building and compiling the part of lsyncd written in Lua
message(">>> Using LUA v" ${LUA_VERSION_STRING} " LUA_INCLUDE_DIR:" ${LUA_INCLUDE_DIR} " LUA_LIBRARIES:" ${LUA_LIBRARIES} " LUA_EXECUTABLE:" ${LUA_EXECUTABLE} " LUA_COMPILER:" ${LUA_COMPILER})
Run cmake this way:
# cmake .. -DLUA_INCLUDE_DIR=/usr/include/lua-5.3 -DLUA_LIBRARIES=/usr/lib64/liblua-5.3.so -DLUA_EXECUTABLE=/usr/bin/lua-5.3 -DLUA_COMPILER=/usr/bin/luac-5.3
You should see:
[...] -- Found Lua: /usr/lib64/liblua-5.3.so (found version "5.3.4") >>> Using LUA v5.3.4 LUA_INCLUDE_DIR:/usr/include/lua-5.3 LUA_LIBRARIES:/usr/lib64/liblua-5.3.so LUA_EXECUTABLE:/usr/bin/lua-5.3 LUA_COMPILER:/usr/bin/luac-5.3 [...]
If everything is ok we can try:
# make
And then:
# make install
This should end installing the following binary: /usr/local/bin/lsyncd
.
Now for the good Lord I'm lazy to research how to create the systemctl lsyncd.service
so being lsyncd a simple binary I just want to replace it with the proper, newer version.
So I want to install the RPM of stock lsyncd:
# yumdownloader lsyncd
I install the rpm with –no-deps
because sometimes I compile rsync
and the stock rsync
RPM is a dependency of lsyncd… so I don't want it installed over the newer version.
# rpm -ivh --nodeps lsyncd-2.2.2-1.el7.x86_64.rpm
This will install the service and default config.
# lsyncd --version Version: 2.2.2
Now we can just backup the unwanted lsyncd binary at /usr/bin
:
# cp /usr/bin/lsyncd{,.bak}
And move the 2.3.1 binary from /usr/local/bin
into the bad, old binary:
# mv /usr/local/bin/lsyncd /usr/bin/lsyncd
Check version:
# lsyncd --version Version: 2.3.1
And now we can edit /etc/lsyncd.conf
, then
Use service lsyncd start
and service lsyncd status
to check and setup everything.
Logs are at /var/log/lsyncd
You're welcome.
Since lsyncd config files are LUA scripts, you can do something like this in a loop, configuring _my_sites
for example:
Example conf file:
settings { logfile = "/var/log/lsyncd/lsyncd.log", statusFile = "/var/log/lsyncd/lsyncd.stat", statusInterval = 60, nodaemon = false, insist = true -- maxProcesses = 8, -- will not spawn more than these number of processes. This adds across all sync{}s -- -- but when using default.rsyncssh must have maxProcesses set to 1 -- maxDelays = 1000 -- when this amount of delayed events is queued, actions will be spawned, even below the delay timer. } _my_dst_host = "user@host" _my_sites = { { host=_my_dst_host, source="/path/to/sync/", targetdir="/path/to/target/", delay=5 }, { host=_my_dst_host, source="/path2/to/sync/", targetdir="/path2/to/target/", delay=5 } } for _idx, _site in pairs(_my_sites) do sync { default.rsyncssh, host = _site['host'], source = _site['source'], targetdir = _site['targetdir'], delay = 5, -- default will aggregate events up to delay seconds or 1000 separate uncollapsible events, which ever happens first delete = true, -- exclude = {'/**/th'}, -- exclude thumbnail dirs. rsync = { compress = false, archive = true, -- chown = "user:group", delete_excluded = true }, ssh = { identityFile = "/path/to/.ssh/id_rsa_host", port = 22, _extra = {"-o StrictHostKeyChecking=no"} } } end
You can check other neat tricks here at the FAQ of lsyncd manual:
~~DISCUSSION~~