Actually the relevant part of the immigration rules state :-
the applicant is married to or the civil partner of a person who has a right of abode in the United Kingdom or indefinite leave to enter or remain in the United Kingdom and is on the same occasion seeking admission to the United Kingdom for the purposes of settlement and the parties were married or formed a civil partnership at least 4 years ago, since which time they have been living together outside the United Kingdom
From what you post you were first living together in the UK, but for the past 2 years have lived together in Ireland? If that is the case then the words "since which time" ... which I read as the date of marriage .... you have not lived together outside the UK, only for part of that time.
You would of course need to pass the UK's Life in the UK Citizenship Test ..... before making the application ... but even if you do, I don't think it is at all clear that the spouse visa issued to you would be in the ILE form.