Re: [luv-talk] remote list of changed files
On Thu, 24 Jan 2013, Andrew McGlashan <andrew.mcglashan@affinityvision.com.au> wrote:
Whilst I did learn a thing or two with this thread, thank you very much, I am astounded that you can seem to have no trouble reading Wikipedia articles to attack people, but you can't seem to read a man page?
Andrew, your problem is that you seem to believe that there are no facts and that anyone who disagrees with you is attacking you personally and rejecting your opinion. However in the real world there are facts and we should all try to discover them not just make things up. Wikipedia is really easy to read, that's one advantage of having lots of people read and edit it. A common difficulty with Unix man pages is that the terms which are used by the program differ from the plain English terms that some users use, this makes a search of the man page unlikely to find the desired answer. A man page which is more than 3000 lines long is difficult to read at the best of times, the rsync man page is about the length of a 9 page magazine article! Also you don't get to criticise people for having "trouble reading" after you cited a conspiracy theory site and said "I'm not going to verify it's correctness or otherwise, please don't ask me to". If you aren't going to try to verify that your references are correct (or even sane) then you have no right to criticise anyone else.
It's not the first time something so basic about a Linux tool or operation has escaped your vast experience and that really surprises me.
Linux and the software which runs on it is actually very complex. No-one can know everything about how it works. For example when I was at university I spent dozens of 8 hour days reading man pages to learn about Unix (mostly Solaris and HP-UX) and still only learned a fraction of it (I did have professors and post-grad students start asking me for advice on Unix issues though).
The other most significant oversight from you was the checkarray processing of software RAID. Just the same, interesting, I'm sure you are a very knowledgeable Linux user, but sometimes I just wonder in total amazement.
If you want to demonstrate your skill then you could try answering questions on the main LUV list. You could offer to give lectures on topics that you know well. You could write blog posts which teach people about things related to Linux and have them syndicated on Planet Linux Australia. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
Russell Coker wrote:
A man page which is more than 3000 lines long is difficult to read at the best of times, the rsync man page is about the length of a 9 page magazine article!
It's 43 A4 pages here, when rendered thusly: man -Tps rsync | ps2pdf - rsync.pdf Output quality varies between manpages, but hopefully the lurkers appreciate learning that little trick. As you say, manpages are generally a reference material, not a HOWTO guide -- they work best when you already know what you're looking for, and have just forgotten the specifics details.
Hi, On 24/01/2013 2:12 PM, Trent W. Buck wrote:
Russell Coker wrote:
A man page which is more than 3000 lines long is difficult to read at the best of times, the rsync man page is about the length of a 9 page magazine article!
It's 43 A4 pages here, when rendered thusly:
man -Tps rsync | ps2pdf - rsync.pdf
Output quality varies between manpages, but hopefully the lurkers appreciate learning that little trick.
Yes, a nice little "trick", but you need ghostscript installed to get ps2pdf -- don't know if it is available in any other packages. Also, on Unix systems you can usually press the "v" character on the keyboard and you have vi or vim open up with the page. A little substitution (search and replace) and it can be readable. man --help|grep ascii -7, --ascii display ASCII translation of certain latin1 chars Extending on this you could simply do: man -7 rsync > rsync.txt wc -l rsync.txt 2079 rsync.txt And with many Linux tools, you can do a quick help as follows: rsync --help sync --help|wc -l 144 Usually there is much more concise output.
As you say, manpages are generally a reference material, not a HOWTO guide -- they work best when you already know what you're looking for, and have just forgotten the specifics details.
Yes, but they also usually have some examples and Google isn't a bad search engine, even if Google is an advertising company... ;) Nonetheless, the other interesting information from the thread was great! -- Kind Regards AndrewM
On Thu, 24 Jan 2013, Andrew McGlashan <andrew.mcglashan@affinityvision.com.au> wrote:
man --help|grep ascii -7, --ascii display ASCII translation of certain latin1 chars
Extending on this you could simply do:
man -7 rsync > rsync.txt
wc -l rsync.txt 2079 rsync.txt
The -7 option to man made no difference to the number of lines reported in my test. $ man -7 rsync | wc -l 3446 $ man rsync | wc -l 3446 I'm using Debian/Wheezy with rsync 3.0.9. I guess you have an older version. 3446 lines was about 9 pages by my quick estimation. $ man rsync | wc -w 24206 Also based on word count it would be as much as 13 magazine articles assuming that what wc regards as a word is a good match for what a magazine regards as a word. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
On 24/01/2013 3:08 PM, Russell Coker wrote:
The -7 option to man made no difference to the number of lines reported in my test.
No, but the plain text format _might_ be more useful and the output can be easily viewed and searched using your favourite text editor.
I'm using Debian/Wheezy with rsync 3.0.9. I guess you have an older version.
I am using Debian stable, so my version will be different. - actually 3.0.7
$ man rsync | wc -w 24206
Don't forget about the less verbose and usually more easily digestible option that I mentioned. rsync --help The word count if that interests you (even though my version is different to yours) rsync --help|wc -w 975 And the line count is still 144, even though I had a typo in my earlier email. Cheers A.
On Thu, 24 Jan 2013, Andrew McGlashan <andrew.mcglashan@affinityvision.com.au> wrote:
On 24/01/2013 3:08 PM, Russell Coker wrote:
The -7 option to man made no difference to the number of lines reported in my test.
No, but the plain text format _might_ be more useful and the output can be easily viewed and searched using your favourite text editor.
$ man rsync | md5sum 4c87c9384b588344f28bb78837e8e478 - $ man -7 rsync | md5sum 4c87c9384b588344f28bb78837e8e478 - On my system they both give the same result. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
On 24/01/2013 4:08 PM, Russell Coker wrote:
$ man rsync | md5sum 4c87c9384b588344f28bb78837e8e478 - $ man -7 rsync | md5sum 4c87c9384b588344f28bb78837e8e478 -
On my system they both give the same result.
Okay, well it might depend on the characters used. Other man pages may or may not be the same. Piping the output _may_ have an effect, but I don't think so. Cheers A.
On 24/01/2013 4:08 PM, Russell Coker wrote:
$ man rsync | md5sum 4c87c9384b588344f28bb78837e8e478 - $ man -7 rsync | md5sum 4c87c9384b588344f28bb78837e8e478 -
fwiw, there is a difference in the man page itself... $ man man |md5sum bc55bd1cc30d654fef7b098fa6af1967 - $ man -7 man |md5sum 8a924afeb13c1d76560f8c43a6fc6cd4 - Cheers A.
Andrew McGlashan wrote:
On 24/01/2013 3:08 PM, Russell Coker wrote:
The -7 option to man made no difference to the number of lines reported in my test.
No, but the plain text format _might_ be more useful and the output can be easily viewed and searched using your favourite text editor.
If your editor cannot handle Unicode, you need a better editor. Or export LC_ALL=C to "fix" all your programs once and for all. As to searching, you have a valid point, which is why Debian maintainers are encouraged to correct upstream on your behalf: http://lintian.debian.org/tags/hyphen-used-as-minus-sign.html
Dear Russell, On 24/01/2013 1:54 PM, Russell Coker wrote:
On Thu, 24 Jan 2013, Andrew McGlashan <andrew.mcglashan@affinityvision.com.au> wrote:
It's not the first time something so basic about a Linux tool or operation has escaped your vast experience and that really surprises me.
I was talking about a couple of the most most commonly used Linux tools here; ones that I would have thought you would have been far more well informed. I know you have great experience, but sometimes it is lacking and astonishing to say the least. Cheers -- Kind Regards AndrewM
On Thu, 24 Jan 2013, Andrew McGlashan <andrew.mcglashan@affinityvision.com.au>
Whilst I did learn a thing or two with this thread, thank you very much, I am astounded that you can seem to have no trouble reading Wikipedia articles to attack people, but you can't seem to read a man page?
It's not the first time something so basic about a Linux tool or operation has escaped your vast experience and that really surprises me.
linux is complex. as are many of the command line tools and there are thousands of them (with perhaps only a few dozen being so generically useful that they're used daily). it is impossible to know every thing there is to know about linux or the software that runs on it (incl. knowledge about either the existence of particular tools or of some of the more specialised/obscure options and capabilites that they have). i've been using unix and linux for over 20 years - so long that i barely remember what computers were like pre-linux. I consider myself to be an expert in several areas of linux knowledge and practice and profoundly ignorant in others. Just like Russell, and just like the handful of people on this list that I consider to be near, at, or above my own level of skill (for the record - you're not one of them), and just like every other highly capable member of the linux community. I'm still learning new things and new variations of things I already knew all the time - it's one of the things i like best about linux...both the continuous discovery/learning process and the pleasant surprise when I need to do something new or unusual and realise that someone else already thought of that and there's a tool or option to do it (or something very like what I need which can do the job with a little hacking - or, at worst, borrow their ideas/methods and adapt them) i've been using rsync routinely for over 15 years, at least since it was first packaged for debian in 1996 and i had no idea that it had batch capabilities. it's not a feature i've ever needed. like a lot of software, rsync has many specialised capabilities that are only of interest in particular use-cases. i knew about --dry-run (use it all the time, and it's a very useful option on several other tools including patch), but not about --itemize-changes, which sounds like it might be useful, as i tend to use --verbose and redirect output to a log file anyway (for viewing with less or tail -f or grepping). so, i'm not in the least bit surprised that Russell didn't know about some of rsync's options. it's not unusual, or even worth comment on. craig -- craig sanders <cas@taz.net.au>
Craig Sanders <cas@taz.net.au> wrote:
linux is complex. as are many of the command line tools and there are thousands of them (with perhaps only a few dozen being so generically useful that they're used daily). it is impossible to know every thing there is to know about linux or the software that runs on it (incl. knowledge about either the existence of particular tools or of some of the more specialised/obscure options and capabilites that they have).
As evidence of complexity, consider the size of Linux in a Nutshell (6th ed., O'Reilly, 2009) - 944 pages, according to amazon.com. Bear in mind that this book summarizes only widely used/available commands and options; it falls a very long way short of completeness. The desired command or option for a given task is not always easy to find, and there are often multiple solutions to a problem, which is one reason why communities such as users' groups are valuable, and why most contributors are likely to learn something new, at least occasionally, from the efforts of participants to help others.
On Fri, 25 Jan 2013, Craig Sanders wrote:
so, i'm not in the least bit surprised that Russell didn't know about some of rsync's options. it's not unusual, or even worth comment on.
Heck, we're rsyncing from NFS servers to other NFS servers, and are seeing problems on the clients (actually related to an ext3 bug in RHEL5, it turns out. They've gone silent on us in the past few days, so I fully expect them to come out in the next few days to prematurely EOL RHEL5 just so they don't have to apply a one line patch[1]). Did you know rsync's -O option is documented in the manpage as being recommended for NFS servers (no explanation as to why - you've got to read quite a few bugzilla entries for that to become apparent)? No, we didn't either. Still, we're preferring that the original bug gets fixed instead. [1] I've never actually seen a RHEL bug successfully closed before that ended something other than "WONTFIX EOL". -- Tim Connors
Quoting Tim Connors (tconnors@rather.puzzling.org):
[1] I've never actually seen a RHEL bug successfully closed before that ended something other than "WONTFIX EOL".
Ah, so, so very familiar. http://www.jwz.org/doc/cadt.html (I'm friends with Luis Villa, who was the poor sod who closed Jamie's bug, thereby earning his ire.) -- Cheers, WebMD is like a Choose Your Own Adventure book Rick Moen where the ending is always cancer. rick@linuxmafia.com -- Kelly Oxford McQ! (4x80)
On Thu, 24 Jan 2013, Rick Moen wrote:
Quoting Tim Connors (tconnors@rather.puzzling.org):
[1] I've never actually seen a RHEL bug successfully closed before that ended something other than "WONTFIX EOL".
Ah, so, so very familiar. http://www.jwz.org/doc/cadt.html
(I'm friends with Luis Villa, who was the poor sod who closed Jamie's bug, thereby earning his ire.)
I frequently remind my boss that I've had many bugs successfully closed in debian, which also incidentally, we don't pay tens of thousands of dollars per year for support. -- Tim Connors
On Fri, Jan 25, 2013 at 03:15:23PM +1100, Tim Connors wrote:
[1] I've never actually seen a RHEL bug successfully closed before that ended something other than "WONTFIX EOL".
yep, that's what you pay the big bucks support contracts for...the peace of mind you get from timely, effective, and above all helpful support. craig -- craig sanders <cas@taz.net.au> BOFH excuse #388: Bad user karma.
On 25/01/2013 8:32 AM, Craig Sanders wrote:
so, i'm not in the least bit surprised that Russell didn't know about some of rsync's options. it's not unusual, or even worth comment on.
It's not that he didn't know, but that he didn't apply the basic application of well known methods to find the answer himself. That is, he couldn't pick up on the very simple "--help" commandline option or how to read a man page; even if he had to texturize the output and do a simple search. Cheers A.
On Sat, 26 Jan 2013, Andrew McGlashan <andrew.mcglashan@affinityvision.com.au> wrote:
It's not that he didn't know, but that he didn't apply the basic application of well known methods to find the answer himself. That is, he couldn't pick up on the very simple "--help" commandline option or how to read a man page; even if he had to texturize the output and do a simple search.
I searched the man page (no-one reads through something that long) but obviously didn't use the right search terms. I could probably have spent more time on a Google search and found the answer. But I don't think it's a problem to discuss these things on the list. Several different solutions to similar problems were mentioned, probably many people who read the list learned things that they will use to solve different problems. As long as it's generally educational I don't think there's a problem in such a discussion. In terms of what's best for LUV it seems that telling people not to ask what might be considered basic questions is a bad thing. It stops people from learning. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
Hi, On 27/01/2013 12:05 AM, Russell Coker wrote:
I searched the man page (no-one reads through something that long) but obviously didn't use the right search terms.
Okay, but surely you must have known the answer was probably there. Perhaps a better solution would have been to find the "simple" solution, which is clearly "--dry-run" and then bring this to the list and elaborate on your requirements and see if this is the best answer or if there is a better or useful other option to consider. That way you don't have people coming back to you and say something like RTFM or provide a lmgtfy link. As it turned out, the topic was quite interesting -- the side topic was not, but you are an attack dog, so we can expect nothing other than that from you. Kind Regards AndrewM
On Sun, 27 Jan 2013, Andrew McGlashan <andrew.mcglashan@affinityvision.com.au> wrote:
not, but you are an attack dog, so we can expect nothing other than that from you.
Wow! -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
Hi, On Sun, Jan 27, 2013 at 1:49 AM, Russell Coker <russell@coker.com.au> wrote:
On Sun, 27 Jan 2013, Andrew McGlashan < andrew.mcglashan@affinityvision.com.au> wrote:
not, but you are an attack dog, so we can expect nothing other than that from you.
Wow!
Russell, I detect surprise at such a comment?
Frankly, and as kindly as possible, need to point out that IMHO, since your return from Europe in the late 90's, you have been very opinionated on this list. Sometimes, almost to the point of bullying....you only reap what you sow. Granted, you have mellowed over the years, but I invite you to sit down for a couple of hours and re-read your comments over the last almost decade and a half on these lists. There are thousands ... but believe me...it could be most enlightening from a personal development POV. RTFM was the flavour of your most ardent comments very early on. BW
thelionroars wrote:
Pfft; easy. I would struggle with traditional cpio, though. Or nc (either version) or socat. Incidentally, the comic neglects to take into account that (GNU) tar has gained --xz in the last decade.
On 01/02/2013, at 18:18, "Trent W. Buck" <trentbuck@gmail.com> wrote:
thelionroars wrote:
Pfft; easy. I would struggle with traditional cpio, though. Or nc (either version) or socat. Incidentally, the comic neglects to take into account that (GNU) tar has gained --xz in the last decade.
most every version of tar on any modern systems I've used recently don't even require you to specify the compression when extracting. just tar -xf filename will suffice
hannah commodore wrote:
On 01/02/2013, at 18:18, "Trent W. Buck" <trentbuck@gmail.com> wrote:
thelionroars wrote:
Pfft; easy. I would struggle with traditional cpio, though. Or nc (either version) or socat. Incidentally, the comic neglects to take into account that (GNU) tar has gained --xz in the last decade.
most every version of tar on any modern systems I've used recently don't even require you to specify the compression when extracting. just tar -xf filename will suffice
For files, but not for streams (IIRC). I've mostly been using mksquashfs instead of tar for the last few years, because it preserves more metadata by default, offers better compression, and the result can be treated either as an archive (unsquashfs) or a filesystem (mount). On pre-squashfs systems, it builds out of git and runs fine without installation, only needs zlib1g-dev and optionally xz-dev. Craig, re run-without-args -- I like it!
On 2/02/2013 10:57 AM, Trent W. Buck wrote:
I've mostly been using mksquashfs instead of tar for the last few years, because it preserves more metadata by default, offers better compression, and the result can be treated either as an archive (unsquashfs) or a filesystem (mount). On pre-squashfs systems, it builds out of git and runs fine without installation, only needs zlib1g-dev and optionally xz-dev.
What do you lose with tar when ran as root that you don't lose with mksquashfs ? Using mksquashfs looks like a handy way to archive data and to be able to quickly mount it as a file system for view only requirements. Thanks A.
Andrew McGlashan wrote:
On 2/02/2013 10:57 AM, Trent W. Buck wrote:
I've mostly been using mksquashfs instead of tar for the last few years, because it preserves more metadata by default, offers better compression, and the result can be treated either as an archive (unsquashfs) or a filesystem (mount). On pre-squashfs systems, it builds out of git and runs fine without installation, only needs zlib1g-dev and optionally xz-dev.
What do you lose with tar when ran as root that you don't lose with mksquashfs ?
I don't remember offhand; possibly nothing. Glancing at the manpages, sparsefiles and xattrs appear to be possibilities. Incidentally, I got burned recently by rsync -a not copying xattrs, which meant that a web server lost its cap_net_bind_service=ep and wouldn't start up. mksquashfs defaults to packing as a tarbomb, but also defaults to extracting into ./squashfs-root (and refuses to act it if already exists), which is annoying for quick-and-dirty copies - for which I would usually use rsync or tar. One ridiculous gotcha is that squashfs regexp exclusion is per-dir, so mksquashfs x x.sq -regex -e usr/share/info/ actually needs to be written mksquashfs x x.sq -regex -e '^usr$/^share$/^info$/' ...otherwise it will exclude e.g. /usr/share/zoneinfo/. Also, mksquashfs reqires all arguments AFTER the sources and target; unsquashfs requires them all BEFORE. AFAIK -b 1M -comp xz -Xbcj x86 -Xdict-size 100% will give best compression (if you are including e.g. ARM binaries, add to -Xbcj). It does block dedupping and stuff. I thought about repeatedly appending to a squashfs rather than using rsnapshot, but it was all a bit too fiddly (-root-becomes ... is yuk; when it gets too big, you can't easily delete only the old snapshots).
On 3/02/2013 11:38 AM, Trent W. Buck wrote:
Andrew McGlashan wrote:
On 2/02/2013 10:57 AM, Trent W. Buck wrote:
I've mostly been using mksquashfs instead of tar for the last few years, because it preserves more metadata by default, offers better compression, and the result can be treated either as an archive (unsquashfs) or a filesystem (mount). On pre-squashfs systems, it builds out of git and runs fine without installation, only needs zlib1g-dev and optionally xz-dev.
What do you lose with tar when ran as root that you don't lose with mksquashfs ?
I don't remember offhand; possibly nothing. Glancing at the manpages, sparsefiles and xattrs appear to be possibilities. Incidentally, I got burned recently by rsync -a not copying xattrs, which meant that a web server lost its cap_net_bind_service=ep and wouldn't start up.
Okay, then perhaps "rsync -aX" might have fixed your problem with the web server. This from rsync man: -X, --xattrs preserve extended attributes Now I am wondering if I should add -X option to my rsnapshot conf files. Thanks A.
On Sat, Feb 02, 2013 at 10:57:08AM +1100, Trent W. Buck wrote:
Craig, re run-without-args -- I like it!
except it turns out i was wrong - both tar and cpio now just print a very brief usage message and have a non-zero exit code (2) if run without args. craig -- craig sanders <cas@taz.net.au>
On Fri, Feb 01, 2013 at 06:18:12PM +1100, Trent W. Buck wrote:
Pfft; easy. I would struggle with traditional cpio, though. Or nc (either version) or socat. Incidentally, the comic neglects to take into account that (GNU) tar has gained --xz in the last decade.
also, just 'tar' by itself with no options or args is a valid tar command, which should disarm the bomb. (some people just like to complicate things) craig ps: same for cpio. who needs args when you have stdin/stdout :) -- craig sanders <cas@taz.net.au> BOFH excuse #365: parallel processors running perpendicular today
Dear Craig, On 25/01/2013 8:32 AM, Craig Sanders wrote:
level of skill (for the record - you're not one of them), and just like every other highly capable member of the linux community.
You don't know what I am or am not capable of because I usually solve my own problems. Cheers A.
On Sat, 26 Jan 2013, Andrew McGlashan <andrew.mcglashan@affinityvision.com.au> wrote:
You don't know what I am or am not capable of because I usually solve my own problems.
I think that people get judged by the questions that they answer not by the questions that they ask. If you want people to believe that you have good technical skills then you might try to be the first person to answer some of the more difficult questions that are asked on luv-main. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
participants (10)
-
Andrew McGlashan -
Brent Wallis -
Craig Sanders -
hannah commodore -
Jason White -
Rick Moen -
Russell Coker -
thelionroars -
Tim Connors -
Trent W. Buck