I am afraid you are right to be concerned, and I think HMRC is wrong.
If I have understood your query correctly, your property was worth £120,000 and you sold it to your son for £60,000. It cost you £84,000 roughly 7 years ago.
If you had sold it on the open market, you would expect to have made a capital gain of £36,000. (That is what the market valuation of £120,000 implies).
According to tax law:
The fact that you made a "real" capital loss is irrelevant.
You are deemed to be "connected" with your son.
Therefore, you are obliged to account for CGT as if you had sold it at market value, regardless of "what you actually sold it for".
HMRC is basically entitled to its slice of the CGT it SHOULD have had, regardless of your generosity.
As you have been deemed to sell the property for proceeds of £120,000, then your son will be deemed to have acquired the property for £120,000 - not the £60,000 he actually paid for it. (That is some small comfort, but to your son, not to you).
You may be eligible to offset your CGT Annual Exemption against the deemed gain (assuming you made no other capital disposals in the tax year against which you would rather it were offset).
If you owned the property jointly, say with your spouse or civil partner, then they might also be able to offset their own Annual Exemption.
If you have made any capital improvements to the property within your period of ownership, then these costs may also be offset against the gain (assuming they were not repairs, or not otherwise allowed against rental income during your ownership).
If it is a residential property, then the gain will be taxable at either 18% or 28% depending (broadly) on whether or not you are a Higher Rate taxpayer already - or the taxable gain puts you in the Higher Rate bracket.
If you occupied the property as your main residence during that period, then a corresponding proportion of the gain may be exempt - "Only or Main Residence Relief" (but the cost of any improvements may be restricted accordingly).
One last point: assuming the property is a UK residential property sold after 5 April 2020, you should have notified HMRC and made a payment on account of the CGT estimated to be due, within 30 days of sale/disposal to your son. This new rule applies regardless of whether you are already in Self Assessment and filing tax returns, or not. There are penalties for such returns made late. However, you should appeal any penalties, on the basis that HMRC has done a frankly abysmal job of warning taxpayers about the new regime.