Anytime your profile does not fit neatly into an IRS "pigeonhole", they get suspicious. And if they find something wrong, and can raise a question like "Fraud?" they can go back TEN YEARS, not just three but TEN, and ask for proof of everything. Wait, you didn't keep ten years of paperwork trails? Ooops....
Sometimes they'll simply ask question and accept a reasonable answer. Sometimes, someone is on a crusade.
Since the OP hasn't told us everything...his best bet is to find an EA (Enrolled Agent) or CPA with a business tax practice, spend a hundred bucks or two on an interview, and get some advice. Tax preparation expenses are normally deductible, and the EA and CPA both have malpractice insurance
in case they've steered you wrong.
Also, the IRS tends to be way less cranky when you can say "See? Here's the letter from my tax pro that told me I should do this." That tends to cut the fines and penalties, big time. And if there are special rules for your type of employment
? The pros are the ones who should know about them.