this post was submitted on
87 points (80% like it)
115 up votes 28 down votes
shortlink:
to anything interesting: news article, blog entry, video, picture...

86
87

Just lost some user data, done with this job, put in my two weeks (self.sysadmin)

submitted ago* by [deleted]

It's been nice reading you guys. I've been looking for a good reason to leave my job and after yesterday, I think I'm ready.

My work, we host a $3mil application called Skyward. It's a Student/HR/Finance management suite and we host it for about 1/3 of the state of Florida. It uses ancient database technology which has its ups and downs.

Anyway, yesterday, first thing in the morning, the entire system was unavailable due to some crashed databases. Not having the best training in this environment, I ran some commands to stop all of the Progress software on the database server and restart it all. 2 hours later the system was back up.

Well, that process broke our replication of said databases (20 of them). When replication breaks, transaction log files build up until you run out of disk space. Given the speed at which the files build up, and the sheer number of databases, I couldn't disable replication on all of the databases fast enough and 3 of them crashed again.

2 of them were recoverable via some expensive late-night support from the company itself, however one of the student databases lost an entire days worth of work/input.

I'm done, I've had it. This software has already put a small handful of people in the hospital for stress related ailments, it's put some of our staff in the hospital for similar reasons. We've had 3 people quit because of this application, several people in the individual districts have quit, and one of our districts personnel even had a heart attack. It's given me a 2/3 full head of grey hair at 30 years old (when previous to this job, I had none).

This job makes me physically sick to my stomach just thinking about going into work each day. I just sent in my two weeks notice.

Luckily my upcoming wedding and honeymoon are already paid for....

all 103 comments
sorted by:
best
hot
new
controversial
top
old

[–]castufariIT Manager 41 points42 points ago

Sad to see someone leave under these conditions but no amount of money is worth that type of gig. I left a job myself due to some crappy software. The medical director loved it but it was a pain in the ass. When a call to support results in them calling in a retired guy to fix it, yeah, it's over.

Enjoy your time off and good luck.

[–]mxpower 26 points27 points ago

I wouldnt quit until I had something lined up. Its a rough world out there currently.

[–]MrGravyMSP 3 points4 points ago

Agreed. Hope the market is good in Florida...

[–]MonsieurOblongSenior Systems Engineer - Unix 0 points1 point ago

The tech industry in general is strong, but Florida has one of the highest unemployment rates in the country I think.

[–]RobotPirateMonkey 7 points8 points ago

Good riddens to that, and I wish you a fair wind into your next harbor.

It's a lesson that is tough to learn, and I had to forcefully get knocked down a peg before it hit me. We hold on to these positions so tightly, trying desperately not to "fail".

You are not your job. It's just something that you're doing right now from 9-5. You could just as easily be doing something else, and still have the same life/family/hobbies/etc. Oftentimes "giving up" is really just choosing to make your life better, rather than stagnating and muddling through.

[–]theFR34KJack of All Trades 14 points15 points ago

GHahahaha 9-5 are you sure you work in IT?

[–]RobotPirateMonkey 0 points1 point ago

It's just a figure of speech. I paid my dues working nights and weekends for years. Now I tell engineers what to do, go home around 5:30, but still keep my phone on at all times.

[–]castufariIT Manager -3 points-2 points ago

I always have my phone with me. Otherwise I have to get the touchpad if I want to play angry birds. :) /r/firstworldproblems.

[–]dahimi 1 point2 points ago

I work for a university from about 9:30 to 5 or so with lunch. About 35 hours a week or so.

[–]VonSockthroat -2 points-1 points ago

No shit, where do I get that kinda IT job? 12-13 hour days are the norm.

[–]RobotPirateMonkey 10 points11 points ago

If you're working 12-13 hours on a "normal" day...then your boss is doing something wrong. Or you're on a contract team with terrible project planners.

[–]mweathr 4 points5 points ago

In other words, a typical company.

[–]RobotPirateMonkey 2 points3 points ago

Perhaps. At a good firm, the Director of Project Services (or whoever oversees the Project Planners) will see the signs of PMs either overworking their talent or failing to meet deadlines. The first leads to lost engineers, and the second leads to frustrated stakeholders. Either way, management intervention is needed to teach better expectation communication or better timeline management.

Of course, like you implied, not every firm is a good one.

[–]dev_randomJboss/Linux Admin 3 points4 points ago

This. A good company, and more specifically, a good IT director, understands that burnout is the quickest way to drive a shop into the ground. My boss literally WILL NOT let me work more than 40 hours in a calendar week, even in the most extreme circumstances. I've had cases where I had to sneak some work in to get certain things done for various PMs but I don't make a habit of it. If you're working 12-13 hour days, It damn well better be for 3 days with a 4 day weekend ahead.

[–]spifSr. Systems Engineer / Ninja / Lyrical Genius 3 points4 points ago

Skills and self esteem. Get them.

[–]VonSockthroat 0 points1 point ago

Working on the skills. I'm one of the "upper level" techs at a small IT firm. I do my best always and try to leave on-time, but end up getting emergency projects at the end of the day or have to be sent on-site or I'm on-call (which we're not reimbursed for). I exaggerated the 12-hour norm, but it's certainly not unheard of, especially when on-call. Self-esteem...I have it everywhere but work. I've only been doing this for 3 years so maybe I just feel less skilled than a couple people I work with that have been doing it for 10+, perhaps rightfully so.
Currently, I'm learning more about Network Security on my own time, as it's a field that interests me a bit more.

[–]spifSr. Systems Engineer / Ninja / Lyrical Genius 1 point2 points ago

getting emergency projects at the end of the day

Leave earlier. Come in earlier if you have to do so in order to leave earlier.

or have to be sent on-site

so?

or I'm on-call (which we're not reimbursed for)

ask for comp time

As far as the whole years of experience thing...

I know people with less than 5 years of experience who rock. I also know people with 20 years of experience who have been doing the same few simple things for most of that time and are clueless. Focus on getting quality experience, not more of the same over and over again. Generally speaking if you look at the way most people do IT and don't do it that way, you'll be fine. Which is to say, specifically: don't make too many assumptions, don't get set in your ways and get hostile towards the new "fads", troubleshoot like you're Sherlock Holmes or a CSI investigating a crime, etc. Don't be too awed by people who have more experience and think they know everything. IT is a whole new field every few years. Sharks keep swimming all the time.

[–]dahimi 3 points4 points ago

12-13 hour days? I've never worked sustained 12-13 hour days ever.

[–]WickedKoalaSystems Engineer 1 point2 points ago

I'm on my 3rd IT job in 11 years. Each one has been roughly 8-8.5 hour days with the occasional long day. My current boss stresses work-life balance and has no problem pushing back on customers if he knows we're too busy with something else. We do have an on-call rotation but we are paid very generously for it on top of our salary.

[–]ephrion 1 point2 points ago

Government.

[–]gsxrSr dev support and release 4 points5 points ago

You're dead on. Took me a few death march projects to figure out, sometimes you have to simply go home. If the company can't stand how slow a project is going or down time, hire someone or spend some money. I'm over spending my time making up for a company being cheap.

[–]easyjet 5 points6 points ago

Where I used to work, we had a 17 year old app with a 'database' made up of some insanely large text files. (multiple. 100th files) or so. The vendor told us that we were their last customer to be upgraded. Not one of their staff had been at the company when this app was current, and the lead developers who wrote it were dead! They spent 2 years writing the routines for the db upgrade. When they ran it on a copy of ours (we had to literally ship them a NetApp disk array) their testing showed that the upgrade would take, if run continuously, 8 months. Once complete, the database would then be 8 months out of date. Those txns would then need to be upgraded and imported into the upgraded db, that would take about 2 weeks. At which point the new db would be 2 weeks out of date, that would be a 2 day job etc etc. Fucking insane.

My board agreed with the vendor that that was the approach. I quit, I just realised life was too short.

That ws nearly 3 years ago, I found out the upgrade routines have never run longer than 2 weeks before they crap out.. The original data is shit. It'll never happen.

[–]labmansteveI Am The RID Master! 3 points4 points ago

Wow. Definately time to GTFO!

[–]bandman614Standalone SysAdmin 5 points6 points ago

Yep, time to quit. +1

[–]itchyouch 4 points5 points ago

These kinds of things are always opportunities for improvement. G'luck on your future endeavors.

[–]HammerdwarfLinux Admin 0 points1 point ago

Agreed. Leaving a shitty job just means your next one will be better!

[–]matt314159Help Desk 5 points6 points ago

It doesn't sound like it was a good fit for you. If you dread going to work, get out of there. Too late for you I suppose at this point, but my advice would be to strategically plan your exit, don't simply throw up your hands in exasperation one day and say, "F*k you guys, I'm going home!" ...plan ahead, get something else lined up first, then transition out. It's a rough job market out there right now, and there's a growing cadre of companies putting warnings in their ads along the lines of "Unemployed need not apply" "Currently employed applicants, only"

Good luck man.

[–]kualtek 5 points6 points ago

It bothers me so much that 'currently employed applicants, only' is legal. It's really a kick in the face from one company to another and the employee's get the short end of the stick.

[–]matt314159Help Desk 0 points1 point ago

I agree it sucks. But not sure if I would want the unemployed to be considered a "protected class" like it's some kind of disease or disability. It's a buyer's market for labor, and companies who get 500 applicants for a position need to do everything they can to weed that down to the top 10 candidates to interview. I think there's an implicit assumption that if you don't have a job, it may be for a reason, and we don't want to risk it. I don't think it's right, but I think that's where they're coming from.

[–]kualtek 2 points3 points ago

It may make business sense but that doesn't mean it's ethical. At least in the US I feel like companies demand your eternal loyalty with non-compete agreements and NDA's.

I think a lot of companies would be stronger with better hiring practices. The way we do it now can't be the most effective way to find the best candidate.

[–]matt314159Help Desk 0 points1 point ago

agreed on all counts. Eliminating someone from the candidate pool because they're unemployed has huge potential to weed out the best candidate even before she gets an interview.

[–]dev_randomJboss/Linux Admin 0 points1 point ago

I'm not sure it really is legal...

[–]kualtek 0 points1 point ago

I don't know either. It basically makes talent poaching a regular business practice.

[–]rossiFan 4 points5 points ago

No job is worth your health.

[–]Pyro919Jr. Systems Admin 3 points4 points ago

When the options are stress vs. stressed trying to find a new job and homeless you may change your mind.

[–]casfacto 2 points3 points ago

I've had jobs that make me sick in the mornings. After a few months of throwing up on the way to work, and resume updating seems so fun!

[–]daemosWindows Admin 0 points1 point ago

I hope you're joking :(

[–]casfacto 0 points1 point ago

Unfortunately no. It wasn't an IT job, but still did make me sick.

[–]SupraMarioI Think It Was The Google 1 point2 points ago

Holy hell, Good luck on your future endeavors...Like others have said though, you probably should have found another job, the job market is still in the toilet.

[–]layer2routerWelcome, here's your mop 0 points1 point ago

It depends on where you are. Bay Area has a lot of openings, but the cost of living is a killer.

[–]SupraMarioI Think It Was The Google 0 points1 point ago

True, same goes with NYC. I have a friend there who has a job gets good pay, but lives in a hole in the wall and barely survives, but jobs are pretty abundant apparently.

[–]layer2routerWelcome, here's your mop 0 points1 point ago

The jobs are abundant because $65-75k is not enough to get by in SF or NY. Being a sysadmin is hard enough without having to scrape by.

[–]SupraMarioI Think It Was The Google 0 points1 point ago

Yep, I don't make what I would in NY or SF, but my cost of living is 1/10th of what it cost to live in SF/NYC.

[–]mpete510Jack of All Trades 2 points3 points ago

I got to the part where you said Progress and immediately had to post a comment and say sorry to you. My company currently uses them for our software and it is a pain in the ass. We're looking to upgrade out of Progress to something else, likely .NET with SQL backend, but it will take us years and years of blood and sweat to do it.

[–]flerp32s/ms/*nix/g 0 points1 point ago

came here to say something similar. Progress needs to die - every app i've seen built on it is a pile of s#$t.

[–]mpete510Jack of All Trades 0 points1 point ago

I would hope that my company's application isn't a pile of shit... but it does have its share of problems which are because of Progress, most of which wouldn't exist with a .net/*SQL setup.

[–]redditacct 2 points3 points ago*

Many times I have wanted to get up out of my chair during a meeting, go to my car and never return.

The only thing that prevented me was not wanting to screw the other poor bastards who would be left if I did.

I can't imagine having a job where you wake up (after having a full night's sleep) excited to get to work - I used to lay there with my eyes closed hoping for a power outage...

[–]HachyaJack of All Trades 1 point2 points ago

I delay getting to work much of the time, and that is exactly my sentiment as to why. Lone wolf admin here, so it's JUST ME. If I had someone else there to share in what i get to deal with from an admin pov it might be better...

[–]raz0rxx 1 point2 points ago

Good luck to you sir!

[–]IT_SAFEJack of All Trades 1 point2 points ago

Sounds like a good reason to get the hell out. Good luck sir and congrats on getting married!

[–]sunshine_killerSystem Specialist and Programmer 0 points1 point ago*

We just got alios but we will host it for two districts.....wish me luck? They were looking at skyward as well..

[–]redditacct 0 points1 point ago

What does it do - attendance, grades or what?

Seems like it might be an opening for Open Source but I realize that is prob not going to happen.

[–]sunshine_killerSystem Specialist and Programmer 1 point2 points ago

Its the financial software for business admins, so payroll and accounting stuff.

[–]BrettLefty 0 points1 point ago

Hopefully they can find someone to rework their backed software. You said they have 12,000 employees, I believe, so they aught to be able to pony up for some guru to come in and fix their poorly designed software. And as others have said, with the economy like it is, I'm sure it wont take them long to find someone.

[–]emailbitesmyass 0 points1 point ago

DB hosting 12K employees + 160K students

[–]thecircusb0y 0 points1 point ago

Hey man good luck. Today I'm struggling. Just getting tired of being asked to do more and more without less and less time, and being told I can't work early, or late, and I can't get any training funds for software they want me to support.

The only thing that keeps me at work is my debt, and when I'm done paying off debt, I'm done working for the man.

[–]Lord_DreadlowHeadset Hero 1 point2 points ago

This is what I don't like about hosted services and cloud computing. You have no idea what state that server room is in or how well the systems are maintained. If you lose a whole database on your system because you didn't back up or keep your systems maintained, then that is your fault. But if it happens on systems that you have no control over, it may not be your fault, but it is still your problem.

[–][deleted] ago

[deleted]

[–]Lord_DreadlowHeadset Hero 0 points1 point ago

Perhaps. But if I have to backup my own data anyway......However, I do think third party website hosting is a good plan if they provide good bandwidth. I can also see an advantage for a decentralized business with workers spread out over a wide geographical area to use a hosted service for ERP applications and such. I guess maybe I've been thinking inside the box for too long.

[–]HachyaJack of All Trades 0 points1 point ago

Yeah because the stock photos and information hosting providers show you about their DCs are completely accurate.

[–]s32 0 points1 point ago

I can confirm that skyward is a gigantic piece of shit.

[–]daxxruckus 0 points1 point ago

She's still marrying you with the gray hair? haha jk, glad you are moving on to a better job!

[–][deleted] 0 points1 point ago

progress databases should be outlawed.

[–]HachyaJack of All Trades 1 point2 points ago

So should BTrieve...got rid of that sucker, just told them we don't need it anymore, eff off...worked well. :D

[–]tapwater86Sysadmin 0 points1 point ago

I just quit too OP (read some of my history for an understanding why). I wish you luck fellow brother in unemployed life.

[–]MadMonk67Sr. Sysadmin 0 points1 point ago

Man, I bet that's like a huge weight off your shoulders knowing you won't have to deal with that anymore.

Think you'll be able to find a new job pretty easily?

[–]Frenchalps 1 point2 points ago

Really sorry to hear of your situation. Give this a quick read, at 30 don't worry, everything will be alright. http://www.guardian.co.uk/lifeandstyle/2012/feb/01/top-five-regrets-of-the-dying?fb=optOut

Use of this site constitutes acceptance of our User Agreement and Privacy Policy. © 2012 reddit inc. All rights reserved.

REDDIT and the ALIEN Logo are registered trademarks of reddit inc.

π Rendered by PID 19955 on app-143 running c21fb21.