Tiger Lab Migration Part 4: Spotlight Worries
Here's an interesting little gotcha. Not sure if it's a good thing or a bad thing yet.
Probably bad.
In our lab, home accounts are on a server. Or, more accurately, they're on an NFS mount that is shared from a network RAID. The way this works is fairly simple, but kind of tricky. The Macserver handles the authentication for network users who log into client workstations. The clients always have the NFS RAID mounted at /home. When users log in, Macserver specifies their home accounts as /home/username. To make sure the RAID is always mounted, we use a little startup item that's just a very simple automount script to call the RAID and mount it in /home.
There's one other little thing that's weird about our setup, and that's the way the RAID is configured. Our RAID is a very nice, but proprietary system made by a company called Panassas. The way user accounts are created on the RAID is unique, and I don't fully understand it, as I did not set the RAID up, nor do I maintain it. But essentially, from my understanding, each user's home account on the RAID is a separate partition.
Does anyone see the problem here?
Well, I won't keep you in suspense. If you haven't figured it out, here's the problem -- and the more I think about it, the more I realize that it is a problem and not a boon: Whenever a user logs in, a new partition is mounted via NFS. And guess what happens then. You guessed it (or maybe you didn't): Spotlight starts indexing.
Holy fucking shit.
I have 207 users currently on the RAID. They have quotas between 2 and 7 gigs. And they're mostly completely and totally unaware of Spotlight and its idiosyncrasies. If one of them logs in and then, say, shuts down the machine (it will happen, trust me), the Spotlight index will get hosed and the machine will most assuredly begin acting flakey. Or how about this: What if a user logs in, checks his email, then logs out? Then what if another user logs in, does same, and logs out? What if five users come along and do this? Now we've got spotlight indexing five different network mounts on the same machine. What if that machine then gets rebooted before indexing is complete? I shudder to think. Or, what if a user logs in to a machine, indexing begins, she logs out and then logs into another machine? Now Spotlight is indexing the same mount point -- accessing the same database -- from two different machines. I'll say it again: Holy fucking shit.
This is a recipe for disaster.
Fortunately, I am an expert in the various methods for turning off Spolight. And that's what we'll have to do: turn off Spotlight for all 207 mount points. This isn't really that big a deal. One simple command should do it (I hope). But it gets me thinking about all the various other Spotlight related problems we're bound to encounter. For instance, our users do a lot of video, and they're encouraged to use firewire drives for this. Well, firewire drives are indexed as soon as they're mounted. I have no way to change this. What happens if indexing on a firewire drive is stopped (i.e. the user unmounts his/her drive) before it's complete? Now we have a hosed index on the user's firewire drive. Next machine he/she goes to will try to index the drive again, possibly completing the index, possibly not. And, during the indexing period, will performance drop to levels that do not permit video editing? I just don't know. But if they do, it's going to be a real problem. And just how do you educate 200+ users about this? It's way over most people's heads. This is a technology that is supposed to "just work." Unfortunately, in a multi-user, networked environment like ours, my worry is that it will "just break."
I'm feeling very hesitant about this migration. It wouldn't be the first time Apple's plans for the home user have come at the expense of the networked lab user. They often seem to forget about us, even though, in many ways, it's this sort of environment for which OS X is so great. Ironic. But if you think about it, one of the greatest features of Tiger -- Spotlight -- is completely useless in a networked environment. In fact, Spotlight is not even supposed to run on networked volumes. (Why it does on ours, I do not know, though I suspect it's because we're using NFS.) But the firewire thing is really disturbing, and I think really underscores the need for significantly more control over the behavior of Spotlight. If, in the Spotlight Preferences, there were a checkbox for "Disable Spotlight Indexing on External Drives," I'd be the happiest man alive right now.
As it is, I'm just plain worried.
UPDATE:
Another thought occurs to me: Okay, so I disable Spotlight on all 207 accounts. Well, what happens when we create a new account? Spotlight needs to be turned off for that account too. So basically, what this amounts to now, is a script that gets run at least every time a new user account is created -- possibly at every login, just to be safe -- that disables Spotlight on all mounted home accounts.
Oh joy.
Probably bad.
In our lab, home accounts are on a server. Or, more accurately, they're on an NFS mount that is shared from a network RAID. The way this works is fairly simple, but kind of tricky. The Macserver handles the authentication for network users who log into client workstations. The clients always have the NFS RAID mounted at /home. When users log in, Macserver specifies their home accounts as /home/username. To make sure the RAID is always mounted, we use a little startup item that's just a very simple automount script to call the RAID and mount it in /home.
There's one other little thing that's weird about our setup, and that's the way the RAID is configured. Our RAID is a very nice, but proprietary system made by a company called Panassas. The way user accounts are created on the RAID is unique, and I don't fully understand it, as I did not set the RAID up, nor do I maintain it. But essentially, from my understanding, each user's home account on the RAID is a separate partition.
Does anyone see the problem here?
Well, I won't keep you in suspense. If you haven't figured it out, here's the problem -- and the more I think about it, the more I realize that it is a problem and not a boon: Whenever a user logs in, a new partition is mounted via NFS. And guess what happens then. You guessed it (or maybe you didn't): Spotlight starts indexing.
Holy fucking shit.
I have 207 users currently on the RAID. They have quotas between 2 and 7 gigs. And they're mostly completely and totally unaware of Spotlight and its idiosyncrasies. If one of them logs in and then, say, shuts down the machine (it will happen, trust me), the Spotlight index will get hosed and the machine will most assuredly begin acting flakey. Or how about this: What if a user logs in, checks his email, then logs out? Then what if another user logs in, does same, and logs out? What if five users come along and do this? Now we've got spotlight indexing five different network mounts on the same machine. What if that machine then gets rebooted before indexing is complete? I shudder to think. Or, what if a user logs in to a machine, indexing begins, she logs out and then logs into another machine? Now Spotlight is indexing the same mount point -- accessing the same database -- from two different machines. I'll say it again: Holy fucking shit.
This is a recipe for disaster.
Fortunately, I am an expert in the various methods for turning off Spolight. And that's what we'll have to do: turn off Spotlight for all 207 mount points. This isn't really that big a deal. One simple command should do it (I hope). But it gets me thinking about all the various other Spotlight related problems we're bound to encounter. For instance, our users do a lot of video, and they're encouraged to use firewire drives for this. Well, firewire drives are indexed as soon as they're mounted. I have no way to change this. What happens if indexing on a firewire drive is stopped (i.e. the user unmounts his/her drive) before it's complete? Now we have a hosed index on the user's firewire drive. Next machine he/she goes to will try to index the drive again, possibly completing the index, possibly not. And, during the indexing period, will performance drop to levels that do not permit video editing? I just don't know. But if they do, it's going to be a real problem. And just how do you educate 200+ users about this? It's way over most people's heads. This is a technology that is supposed to "just work." Unfortunately, in a multi-user, networked environment like ours, my worry is that it will "just break."
I'm feeling very hesitant about this migration. It wouldn't be the first time Apple's plans for the home user have come at the expense of the networked lab user. They often seem to forget about us, even though, in many ways, it's this sort of environment for which OS X is so great. Ironic. But if you think about it, one of the greatest features of Tiger -- Spotlight -- is completely useless in a networked environment. In fact, Spotlight is not even supposed to run on networked volumes. (Why it does on ours, I do not know, though I suspect it's because we're using NFS.) But the firewire thing is really disturbing, and I think really underscores the need for significantly more control over the behavior of Spotlight. If, in the Spotlight Preferences, there were a checkbox for "Disable Spotlight Indexing on External Drives," I'd be the happiest man alive right now.
As it is, I'm just plain worried.
UPDATE:
Another thought occurs to me: Okay, so I disable Spotlight on all 207 accounts. Well, what happens when we create a new account? Spotlight needs to be turned off for that account too. So basically, what this amounts to now, is a script that gets run at least every time a new user account is created -- possibly at every login, just to be safe -- that disables Spotlight on all mounted home accounts.
Oh joy.
Labels: Lab, Systems, Tiger, TigerLabMigration
This entry was posted
on Wednesday, July 06, 2005 at 7:50 PM.
You can skip to the end and leave a response.
» Post a Comment