The respondent, applicant or recipient may file an answer to the notice within 20 days after service thereof. Answers shall admit or deny specifically and in detail each allegation of the notice, unless the respondent party is without knowledge, in which case his answer should so state, and the statement will be deemed a denial. Allegations of fact in the notice not denied or controverted by answer shall be deemed admitted. Matters alleged as affirmative defenses shall be separately stated and numbered. Failure of the respondent to file an answer within the 20-day period following service of the notice may be deemed an admission of all matters of fact recited in the notice.