Paranoia, for the win…

Today, my team got an email from a local tech. The associate director for his site came to the IT department, saying he was concerned that one of his “tech friends” told him of a rootkit that was supposedly installed on all machines from $WeMakeComputers. And since his site (as well as almost all the other sites my team supports) was almost exclusively using computers from this company, he was worried that $WeMakeComputers could use this rootkit to hack into the facility and take over their computers.

I replied back, explaining to the associate director (by proxy) that, having done several IT equipment refreshes for several sites with contractors from $WeMakeComputers, there was almost no chance whatsoever that the contractors could have installed any such rootkit on the computers they installed while working with our teams in IT. For starters, none of the contractors were given accounts with anything remotely resembling administrative rights, meaning they couldn’t install a mouse on a computer, let alone a rootkit. Also, the software we used to push images to the computers did low-level formats of the hard drives prior to deploying the image onto each computer, which would destroy all traces of pre-installed software. Thirdly, the AV software we used, as well as the network scans, port security, and about two dozen other layers of security, meant that if such a rootkit was installed, it would’ve dealt with it rather quickly. Not to mention the fact that a number of employees and executives at $WeMakeComputers would have faced criminal charges and civil fines for putting such software on computers purchased by a federal government facility. Then there was the matter of the contract, which, among other things, forbade such software from being installed on the systems, meaning if such software was to be found, criminal charges aside, $WeMakeComputers would’ve been in violation of the contract, forcing them to refund us the entire value of the contract, a sizeable amount of their annual income, as well as the potential loss of income if such a story was ever made public, since I doubt many people would take kindly on a major corporation trying to spy on the government.

I also went on to say that I didn’t put much stock in people who heard things from their “tech friends”, because it invariably involved someone’s niece/nephew, grandchild, friend of a friend of a friend, random person they met somewhere in public or at a party, etc., whose credentials and experience in IT support were dubious at best. The irony is, trying to convince these people that the advice from their “tech friends” is wrong far more often than it’s right, or is nowhere near the problem it’s made out to be, is much easier said than done. All too often, such things they hear are based on solely on rumor, supposition, smear campaigns, and flat out lying, and there are no shortage of people out there who’ll believe anything they read on the Internet, and assume that anything written on the Internet MUST be true, because it’s on the Web.

I spent most of the time shaking my head and feeling bad for the local IT guy who emailed us because I’ve been in this local IT guy’s position more times than I could count, trying to correct someone who was a non-tech that heard something from their “tech friend” and assumed it HAD to be true, when this “tech friend” had little to no knowledge or actual experience in IT. It’s one of the more aggravating aspects of being an IT tech. It’s like trying to disprove Judy Patch, when there’s no concensus on whether dear Judy is even real…

Late to the party

Today, my team got forwarded a ticket from a local site that almost made us do a double-take.

The ticket concerned a doctor whose account had been terminated a month ago. All of a sudden, he wanted to have the account reactivated so he could work immediately. There were several problems we discussed…

  1. How could the doctor do his job in the last month, since a terminated account would preclude him from logging in, let alone writing progress notes, or doing anything related to seeing patients?
  2. Why didn’t the doctor call a month ago to have his account reactivated?
  3. The notes on the doctor’s account said that his account was terminated because he failed to complete mandatory training, training which by the way, could get his license suspended for failing to complete.

My coworker, who got the call, tried to contact the doctor, but had to leave a voicemail, and after 2 hours, lowered the ticket to a low priority and kicked it back to the local site to do the work on Monday. He correctly reasoned that it must not have been so “critical to patient care” that the doctor get his account reinstated or to do the required training, if he couldn’t bother to pick up the call of a person actually trying to help him, or call them back within two hours.

More fun the end user virus

So today my now infamous user comes to me and says reports from our msp system are not working at all. So i check it turns out one report she tried to run didn’t work. I asked her if she tried to run any reports today and the answer was no. I said try it and when she did sure enough all but the one report worked.

What is this “logic” you speak of?

Today, a member of my team got a rather amusing call from the Tier 1 Desk Monkeys. The Desk Monkeys noticed that there was an outage for a system, and a minor one at that, at one of our local sites, and wanted to know when we would be putting in the outage notification for it.

To put things in perspective, the last few weeks, the Tier 1 Desk Monkeys have been dumping more and more of their work onto us, in an effort to offload a massive amount of criticism that they (justifiably) got for their horrendous, dare I say hilarious, inability to do many things right, and basically trying to set us up to fail, in a vain effort to prove that it wasn’t just them who were royally screwing things up. This includes the outage notifications they were once in charge of. These notifications were only supposed to go out for things such as maintenance on major IT systems, such as network maintenance for a whole site, the EHR database servers, or for when there was a major outage, such as when a major system crashed, or there was a legitimate outage that needed to be fixed as soon as possible. But the Tier 1 Desk Monkeys were issuing these outage notifications for such menial things as 2 or 3 people being unable to print, a single person having trouble connecting to the network, and even one such notification where someone complained that they couldn’t get onto Facebook, and claimed it was critical to patient care that she be able to do so immediately. And when these notifications went out, up until recently at least, the Tier 1 Desk Monkeys were required to convene a conference call, one in which they demanded that my team, the local site’s IT technicians, upper management and others high up the food chain join, often with little or no notice, and sometimes in the middle of the night, with little or no knowledge of what was going on, or even how we were involved. It was then decided that these conference calls were basically a colossal joke and a waste of time, and no longer required, especially after the avalanche of criticism levied against the Desk Monkeys.

But back to today’s events. The Tier 1 Desk Monkeys called us, and gave us the ticket number for the outage. A cursory lookup of the ticket revealed the local site had already fixed the issue and closed the ticked three hours before the Desk Monkeys called us. The Desk Monkeys didn’t even bother to read the ticket, since it would have been pointless to issue an outage notification, and blast it out on email to everyone and their brother, three hours AFTER the issue was already fixed. When one of my coworkers brought it up to the Desk Monkey who called him and asked him to actually read the ticket, the Desk Monkey’s only response was to cite the new policy to call us for all these issues and insist on an outage report being filed by us. My coworker retorted that it would make no sense to issue an outage report AFTER the issue was already fixed, and for them to insist that a team of people who had nothing to do with the issue in the ticket (let alone a team that didn’t even fix it) issue such a report made even less sense, and if anything, the site itself should have issued the report while the issue was being worked on. Since they didn’t, my coworker reasoned (correctly) that it must not have risen to the level of even needing an outage report in the first place. The Desk Monkey had no response to that and just hung up.

I got a similar call earlier in the day. I was notified that several users across a couple sites were experiencing “slowness” on their computers, but couldn’t be more specific than that. I made the obligatory calls, and got nowhere fast, since no one was picking up their phones. While I was doing this, my coworker was getting IM’d constantly by another Tier 1 Desk Monkey, demanding to know why I hadn’t bothered to put an outage report in yet. My coworker asked me about it, and I told her I simply didn’t have the time yet, because I was trying to actually get a hold of a live human being to get more details on the issue, but the Tier 1 Desk Monkey she was talking to didn’t accept that as an answer, and instead of allowing me to actually work on getting some decent information on the issue that the Desk Monkeys failed to get, demanded the Outage Report Number as soon as possible.

If only to shut up the Desk Monkey and get her off my coworker’s back, I finally did the outage report, and sent it along, and got an email string going, where the issue was quickly discovered to be a slow SAN cluster. A reboot of the cluster caused the “slowness” to disappear and all was well. I closed the ticket and the outage report, but apparently, the Tier 1 Desk Monkeys weren’t done with me yet. I got an IM from the same Desk Monkey pestering my coworker earlier that day, demanding to know why I closed the ticket and the Outage notification out. Rather perplexed, I asked why she needed to know this, and told her to look at the ticket and outage notification, since I had updated both with notes prior to closing them, indicating that the problem was fixed. Her only response was, “Oh… nevermind.”

And the sad thing is, this is fairly typical of the Tier 1 Desk Monkeys. Not only do they not even bother to read the very tickets they create, they cause more work than they do, and even then, they insist on having other people do their work for them, either to try and get them into trouble, or to prevent themselves from getting into trouble…

More idiocy…

Just got another ticket that made me immediately downgrade the ticket…

“User states that a piece from her printer is physically broken after one of her coworkers tried to fix it for her, and she needs it fixed immediately because she cannot do her work without it, and patient care is also affected.”

Even the on-call laughed his head off when he heard the user’s name stating she never sees patients, and the only things she prints are emails she gets from her friends, asking her to meet them to go party somewhere.