Within 20 days after an initial decision becomes a final decision pursuant to §81.104(a) or within 20 days of the mailing of a final decision referred to in §81.104(b), as the case may be, a party may request the Secretary to review the final decision. The Secretary may grant or deny such request, in whole or in part, or serve notice of his intent to review the decision in whole or in part upon his own motion. If the Secretary grants the requested review, or if he serves notice of intent to review upon his own motion, each party to the decision shall have 20 days following notice of the Secretary's proposed action within which to file exceptions to the decision and supporting briefs and memoranda, or briefs and memoranda in support of the decision. Failure of a party to request review under this paragraph shall not be deemed a failure to exhaust administrative remedies for the purpose of obtaining judicial review.