forked from walterdejong/synctool
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathFAQ
218 lines (157 loc) · 8.98 KB
/
FAQ
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
synctool by Walter de Jong <walter@heiho.net> (c) 2003-2012
synctool COMES WITH NO WARRANTY. synctool IS FREE SOFTWARE.
synctool is distributed under terms described in the GNU General Public
License.
See the README for general information.
See the INSTALL file for information on how to deploy synctool.
See the DOCUMENTATION file for information on how to use synctool.
(NOT SO) FREQUENTLY ASKED QUESTIONS
1. I don't understand this tool.
2. Why not use cfengine?
3. Why do you say there is no communication between master and client?
4. I get the error: "rsync: command not found"
5. I get a weird error from rsync
6. I get "command not found" errors
7. I will not allow ssh traffic on my network (for some reason)
8. You can go faster than this: use an MD5 checkum cache for each file,
computed only once on the master node
9. You can go faster than this: do block updates when copying files, like
rsync does
10. Has this tool been tested properly?
11. What about scalability?
12. Have you tried deploying synctool with rsync with multicast?
13. Does synctool not do any versioning or revision control?
14. When was the first release?
15. Does synctool have a logo?
16. Googling for 'synctool' turns up a lot of Microsoft stuff??
1. I don't understand this tool.
Are you a system administrator, administrating a cluster of computers?
Read the README file. Then read the INSTALL file. Try following the steps
described in the INSTALL file. Read the DOCUMENTATION file. Try using it
as described in the examples in the DOCUMENTATION file.
If all else fails, just give up.
2. Why not use cfengine?
cfengine is the main reason why I developed synctool. Need I say more?
It's a great tool, but its learning curve is so steep that many sysadmins
(even capable people) stay away from it.
Personally, I have set up and actively maintained complex cfengine setups.
Still, I like the simplicity of synctool. Making changes in cfengine always
takes more time than doing them in synctool.
A sysadmin tool is supposed to make system administration easier, not harder.
There are things that cfengine can do out of the box that synctool can not,
and therefore some argue that it is more powerful. There are pros and cons
to both. Use whichever you prefer, I know which I like best.
3. Why do you say there is no communication between master and client?
synctool is a python script that reads from the local repository (typically
/var/lib/synctool). Therefore, there is no communication between master
and client, and there is also no complicated protocol or authentication
mechanism for synctool. synctool employs rsync with ssh to copy the masterdir
from the master node to the clients. synctool leaves the tasks of
authenticating and efficiently copying files over the network to the tools
that are fit for this job.
So yes, there is communication, but that part is done by rsync and ssh
and really does not have much to do with the synctool Python program.
The synctool Python program only matches files with certain extensions and
puts them in the right place on the system.
Also note how this setup is safe; it is a server-push setup and never
a client-pull. You can harden your master server with a firewall that
blocks new incoming connections, and still be able to operate synctool
normally, because everything is controlled from the master side.
4. I get the error: "rsync: command not found"
Install rsync from http://samba.anu.edu.au/rsync/
If you already have rsync, edit synctool.conf to make sure the rsync_cmd
setting is correct.
5. I get a weird error from rsync
On some systems, the /usr/bin/rsync executable is an rdist program
rather than the rsync program from http://samba.anu.edu.au/rsync/
Make sure the rsync_cmd setting in synctool.conf is correct.
6. I get "command not found" errors
Make sure the program in the error message is installed.
Check the settings for diff_cmd, ssh_cmd, rsync_cmd, and synctool_cmd
in synctool.conf
7. I will not allow ssh traffic on my network (for some reason)
You will have some means to copy files from one system to another.
The SSH and rsync commands are configurable in synctool.conf, so
change them accordingly.
8. You can go faster than this: use an MD5 checkum cache for each file,
computed only once on the master node
True. But then you no longer can easily fiddle around with the local
repository on a node for testing purposes. Surely there's a way around this,
but at this moment, I think synctool is fast enough and I really like
keeping things simple.
9. You can go faster than this: do block updates when copying files, like
rsync does
True, originally I was even thinking about patching rsync to accept classes
the way that synctool does. This turned out to be harder than it sounds ...
Since synctool is meant to copy small configuration files, there is little
to gain here by doing block updates.
10. Has this tool been tested properly?
synctool has been in use for four years before it went out in the public,
helping the system adminstrators of two Dutch national supercomputer
clusters. During this time, synctool proved to be a very stable and useful
tool.
In the meantime, we migrated to a new system. Again, synctool proved to
be easier, faster and even more reliable than the supplied vendor solution.
On another system with a different architecture, and in a multi-platform
environment, synctool gets the job done.
At the time of the version 4.0 release (August 2009), we are managing
6 clusters of varying sizes and platforms, with synctool.
11. What about scalability?
synctool was developed to help system administration on a small cluster of
big machines. Mind however, that the scalability of synctool lies mostly in
the tool that you use to distribute repositories, for example rsync.
rsync is a great aid to synctool, making it possible to deploy synctool even
on large clusters. I have seen it in action on a 100 node cluster and it had
no problems whatsoever.
Mind the definition of "node" -- the 100 node cluster I'm talking about had
64 processor cores in every node, essentially making it a 6400 node cluster.
A node in synctool terminology is a host running a single system image.
At this moment, the config file is rather clumsy for handling large clusters,
since you have to list each and every node. It is not possible to specify a
range of nodes in a single statement. With some clever scripting around an
'nmap -sP' you can easily map a whole cluster into a synctool.conf file.
There is a script in the contrib/ directory that does exactly this.
On large clusters, it is likely that there will be a lot of output to the
screen. Use -q or --quiet and -a or --aggregate to clean it up.
If you have thousands of hosts, you may want to make a multi-tiered setup
in which you sync to nodes that sync other sets of nodes. It sounds
complicated, but it's quite simple to script when you use synctool groups
in a clever way. For example, make a group of 'syncnodes' that are
synced by the master server. Every syncnode runs synctool-master to their
own subcluster or their own racks. synctool-ssh has an option --no-nodename
to make sure the output comes out right in such a multi-tiered setup.
One other thing, synctool was meant to manage configuration files and not
large databases (for example). If you put a great number (thousands,
millions?) or very large files in the synctool repository, synctool runs
may get slow. For normal use, it should be perfectly OK.
12. Have you tried deploying synctool with rsync with multicast?
No, but I have seen multicast system installations in use with rsync,
and I believe it should be possible to build this around synctool too.
I've also seen problems with rsync in conjuction with multicast, and maybe
you're better off with the smarter bittorrent way of distributing the
repository. I have not experimented with this, but I'm sure it is possible.
13. Does synctool not do any versioning or revision control?
No, but it is possible to deploy it under RCS, CVS, svn, git, or your version
control system of choice. synctool is a simple tool by itself, and happily
delegates the task to a tool that is meant to do version control.
When you do deploy it under a version control system, be sure to ignore any
housekeeping directories in the repository by stating in synctool.conf:
# one of these
ignore RCS
ignore CVS
ignore .svn
ignore .git
By the way, synctool was developed using subversion. Later, I switched to git.
14. When was the first release?
The first release to the public under GPL was synctool 2.2.0 in January 2007.
synctool had already been in use for four years before releasing it to the
public.
15. Does synctool have a logo?
Yes, it's called 'the synctool wrench'. See images/synctool_logo.jpg or
go to the website at http://www.heiho.net/synctool/
16. Googling for 'synctool' turns up a lot of Microsoft stuff??
Sadly, Microsoft named one of their little tools 'synctool', too. I don't
know which one was first, and I don't really mind. I'm confident that
UNIX sysadmins will recognize my synctool when they see it.
I guess you could also google for "cluster synctool".
EOB