first of all thanks to everyone here helping each other

I just wanted some guidance regarding the Pr for an EEA national in the uk
right i am a french who camei n the uk in 2012, got married to a foreigner partner (non EEA)
i was working at that time , she got the extension of her residency document for 5 years as a member of an EEA national.
Few months later i lost my job, i was jobless few month ( in what i never applied for job seeker allowances)
In june 2013 i registered myself as self empoyed and have been since till today.
In between i had 1 baby boy with my partner, we claimed WTC and CTC and offcourd CB.
During these 3 years my income has been on the low side ( roughly 7000 a year), so i was having the wtc and ctc, and steadily but slowly my profit has been going up.
So i am here just to have some guidance, as nearly i am going to apply for the PR for myself and my partner as the member of family of an EEA, can someone guide me, if in my case everytthing should be alright regarding my PR? or do you think because of these 3 years of struggle with my self employement it can be a reason to refuse my PR?
thanks again to anyone reading and helping out
regards