I had to manuver about 800GB of research databases (I use DEVONthink Pro Office) from one zfs filesystem to another because at some point after OS X Server got ahold of my Research volume the permissions and filesystem ACLs went bananas and DEVONthink was completely baffled by it.
Solution was to use Apple’s ignorant-of-xattrs-and-ACLs
1
rsync
to move it to another filesystem where everything is fine. So sometimes it’s good to have a broken
1
rsync
available. Still can’t be certain it won’t happen again and I’d like a more elegant way to recover.
And no, smart-asses,
1
chmod-N-R
did f-all to fix anything. Oh, it ran alright. But
1
ls-le
still listed long ACLs on everything after supposedly removing it. I suspect it’s something to do with posixacls, and ACL inheritance in OpenZFS’s options, or the mimic HFS+ code is involved somewhere.
Either way, I was sweating recovering those databases with long restore times from Google Nearline until I tested one and it was fine again. Going to alter my backup destinations for different databases anyway to rely on S3 for my Personal and Household stuff at least. Oof.
When shell integration is enabled, iTerm2 automatically adds a mark at each command prompt. Marks are indicated visually by a small blue triangle in the left margin.
I started running the test builds of iTerm 3 last week and overall I love it and haven’t had any stability issues that I can recall because of using it (though since I use
1
tmux
most of the time it wouldn’t exactly be a big deal if I did.)
I was a little wary of the idea of shell integration with iTerm 3 but I read about the features this brings and went ahead and started kicking the tires. Automatic profile switching, the ability to right-click a filename on a remote host and
1
scp
a copy of it locally, not to mention easily paging my way through exiting jobs has been a really useful enhancement to an already great piece of software. It’s funny how something like a terminal window can actually improve over time.
(Don’t sweat the fact that the documentation mentions to
1
curl
the script down and then pipe it to
1
bash
for the integration glue; download the script and look it over and you’ll see it figures out you’re using a superior shell like
1
zsh
and correctly appends the appropriate
1
source
directive for you upon execution. Obviously you shouldn’t blindly be
1
curl blah.sh|exec
ing yourself to failure so you’d be checking that script yourself anyway, right?
Liz recently had a conversation going on Facebook where some academic workflows came up. This is the sort of thing I love to read and then run my mouth about, particularly about software and methods of organizing the things that end up getting collected because I think it’s especially relevant to the ABDs.
I had some bookmarks on Pinboard that I hastily shared but I wanted to grab a few more things and put together a more tailored list.
Caveat: Since she and I both are firmly ensconced in the Apple Family, most of these are directly related to iOS and OS X software and I will make no apologies for that. There are many exceptional options for research and organization of information that favor OS X and iOS, and that’s where I spend the majority of my time so my sources are slanted that way.
1
dealwithit.gif
.
The problem I had in putting this list together is that in some cases my bookmarks are a few years old, and things move and change quickly. Because of technology being such a fast-moving target, some of this will be a little different (or a lot different), but this shouldn’t be too distracting since, as best I can tell, most academic researchers commit to something and don’t fiddle with it nearly as much as I do.
General Workflows and Notes for Researchers and Academics
“There’s more than one way to skin a cat, and more than just a few tools for creating a paperless workflow for academic research and writing. I’ve noted some of the possibilities on the Affordable Mac apps for academic tasks page.”
Goes into some detail with various tools like Sente and DEVONthink, including the workflow they use for research, drafting, and polishing their work. Scrivener fan, too.
Luc Beaulieu
The e-Office series has some interesting workflow and notes relevant to academic research.
I love DEVONthink Pro Office so much I want to marry it. Some notes I’ve found dealing specifically with DEVONthink are bound to happen. Luc’s templates are interesting but I’ve got tons of links!
DEVONthink is great at a lot of things, but honestly I don’t find it that exceptional for collecting — especially when I’m away from the computer. I prefer Evernote for that sort of thing.
Go see my friend Patrick, and do not pass Go and do not collect the USD$200. You won’t need it anyway. Paper based markup systems!
Finally
My more complete archive of things I scrape away for later is pretty large, but some especially relevant items include but are not limited to things I tag as:
This is full-frontal nerdery, so if you’re normally here for pictures, links to interesting things, and my never-ending bitching about astroturfed “grass-roots political movements”, this post will probably not be relevant to your interests.
One of the things I’m interested in is log management and intrusion detection, and there are a lot of fascinating options for this sort of thing ranging in cost from hella expensive to free. Any grizzly neckbeard with a three button mouse and a copy of the sed and awk book has probably used logsurfer, tripwire, or any number of other options for keeping an eye on system logs, but I find OSSEC‘s feature set to be far more suitable and flexible and require much less effort and patience.
OSSEC is a really interesting piece of software that manages to do an excellent job at monitoring logs and also acting as a HIDS1. So it is excellent at monitoring system logs, but it also detects changes to system utilities and binaries, and uses some logic that you can interact with to detect anomalous behavior. There are a lot of health monitoring suites that offer to alert system admins based on error messages or changed files, but OSSEC also offers central agent and policy management, active response capabilities, and it works across a variety of network topologies2.
Further information on OSSEC is out of scope so from here on I’ll be assuming you know what it is and that you want to install the agent or the server on a Mac OS X 10.9 Mavericks system and that you’re familiar with homebrew and comfortable with the CLI.
Gotcha 1: llvm — y u no inline asm?
The first gotcha you’ll likely encounter with OSSEC’s install.sh method of building and installing the agent and/or server, is that Apple’s compiler doesn’t like inline assembly language. Oops. You’re going to need to install a new bundle for gcc to proceed, so install one via homebrew.
Ensure sure you have Xcode and the Command Line Tools installed, and then install a version of gcc you’re comfortable with
" rel="footnote">3. Once you’re building you can go for a walk around the prison yard being mindful of inmates with shivs and grudges, and come back to a built and installed
and you’d think you’re going to all square from here on out, but you’d be wrong. Prepare to do battle with what I can only assume is a broken configure script because no matter how I tried to insist on wanting to use gcc/g++ out of homebrew, it refused. Some tutorials will have you putting on a pair of Bad Idea Jeans and replacing Apple’s binaries in
1
/usr/bin/
— and I wave my hands at them disdainfully.
Instead, this will get you going without doing anything too stupid. There is a directory inside your extracted OSSEC tarball called
1
src
and a file in there called
1
LOCATION
. Edit the
1
LOCATION
file and adjust accordingly. Mine merely declares my favored compilers living in
1
/usr/local/bin/
:
1
2
3
4
[crayon-5df11de4e9afa364237831]DIR="/var/ossec"
CC=/usr/local/bin/gcc
GCC=/usr/local/bin/gcc
CLANG=clang
[/crayon]
So now we’ll prefer the stuff we just installed with homebrew over Apple’s tools. You can get fancier with it, but generally I prefer using Apple’s compiler and use vanilla
1
gcc
only when required. You could alternatively set aside Apple’s binaries (as root)
and assume that updates to the Xcode CLI package or OS X can and will stomp these with complete disregard for your feelings.
You may now resume your use of
1
install.sh
to configure and build your server, agent, hybrid, or local instance as expected. While you’re installing agent keys and/or provisioning your Mavericks system for monitoring, you should know that it probably won’t start automatically when you reboot. Weaksauce.
Gotcha 2: StartupItems? What?
OS X doesn’t want to use
1
/System/Library/StartupItems
anymore so we’re not going to try to make it happen. The OSSEC installer shoves some stuff in there and leaves a file in
1
/etc
that isn’t needed, so you can safely remove the OSSEC startup script from
1
StartupItems
because we’re sophisticated users that aren’t running Mac OS X 10.4 or something equally antiquated. Instead we’ll make a
Now I need to load that job and start it, and confirm it started the processes correctly. This won’t work very well if you haven’t actually provisioned the agent for non-server installs, but you can probably find your own way from here.
Easy as pie, piece of cake, something something cupcake. We now have a Mavericks system that compiled and installed OSSEC, and it will start the services upon reboot.
Yes, it’s just that simple.
For more information on OSSEC, please go to the OSSEC website, and if you’re curious about the project and the developer, Daniel Cid, he’s been featured on The Setup a couple of years ago. It’s a pretty good post for The Setup, especially considering it was somewhat unusual seeing a researcher in my field listed there. If you subscribe to the OSSEC mailing lists or read the archives you’ll discover quickly that Daniel and I have many personality traits in common. Guess what they are!
host intrusion detection system, as opposed to network intrusion detection systems like snort ↩
case in point — I’m using the hybrid server mode at home to correlate local event streams and then escalate interesting things to a remote server for further analysis and handling the alerting ↩
What are my favorites? That’s easy: mac-vim, mutt, nmap, dcraw, exiftool, yasm, tmux, sshfs/fuse, gpg, fasd, offlineimap, markdown, mmd, unrar, par2, links, and a few other odds and ends. ↩
Use case: Household web cache to accelerate browsing for all devices and computers, and also improve privacy by configuring Squid to intercept requests destined for analytics providers, and to ensure web browsers are benefiting from other compliance tools like DNS sinkholes to known-malicious addresses.
Benny Kjær Nielsen is conducting an experiment to potentially crowd-fund his brilliant OS X email client MailMate through 2014 and is looking for participants. MailMate is the most capable email client on any platform and is a real powerhouse for anyone that spends a lot of time handling messages. If you are interested in supporting Benny and would like to have your feature requests be given some extra weight, now is an excellent time to consider doing so. Pledges receive licenses and the satisfaction of contributing to this wonderful software!
My favorite email client for OS X is getting better and better with each passing week. First it was an elegant way to get emails into OmniFocus tasks and projects using a hotkey that populates a new entry in the OmniFocus Quick Entry box, and now there is early support for a feature to support Gmail labels as actual tags instead of duplicated folders!
Remarkable! Benny at Freron is kicking ass and not taking any names with MailMate. Some really exciting desktop and mobile email software has been showing up lately and I have some thoughts on damn near all of them, but in the mean time the one you really need to be looking at closely is MailMate. Especially if you use Gmail. And I’ll even go out on a limb and say that if you use
1
pine
,
1
alpine
,
1
re-alpine
or
1
mutt
, it’s also worth a look due to the elegant and highly configuration command set it offers, including drumroll support for different input modes depending on which pane is active.
The only thing I miss from other email software right now is Conversation View with a short headers and synapsis between your folders and message viewer, but now it isn’t worth the compromise to give up everything MailMate does just to get that back by using something else.
Fantastic stuff, Benny. You’re killing it erryday and we all love it.