cis construction revenues claiming money owed

[manning]

whether you are willing or able to advise me i am unsure so here goes,
I have recently been contacted in respect to non compliance of CIS deductions for 2004-2005 - 2006-2007 for three sole traders two have been accepted under conditions 9(4) but one remains and i have been told i will have to pay approx £5,000 back. From what i can gather my bookkeeper put this down as labour paid from the business and the accountant claimed tax deductions although i did not make them. This was because i had a large contract and needed some help ,the person i employed had his own painting business and invoiced me directly when i used him..From what i gather they have accepted the other two because they have filed tax returns for the years concerned but this individuals business name has come up as unknown ? ,i did provide his NI number and UTR number.I thought that as i was not using him regularly and he supplied his own tools ,materials & transport i was OK not to deduct tax at source ,all payments were made through cheques through my business account.
would it be worth me challenging this decision and paying to fight my cause ! or should i just get a loan and pay. any advise would help relief my stress Thanks ,Kind regards,