Neutrophils mediate immunity against both invasive Aspergillus and Candida. (Remember how NADPH oxidase deficiency can cause Aspergillosis? NADPH oxidase pathways exist within neutrophils) A key point to remember is that immunity against mucocutaneous candidiasis is T cell mediated, whereas that against systemic candidiasis is neutrophil mediated. Hence, Chronic Mucocutaneous Candidiasis is a T cell disorder and you do not have fungemia. This explains why there can be systemic Candidiasis or Aspergillosis in neutropenic states.
In most other sites and most other fungal infections, immunity is T cell mediated, hence the increase in lymphocytes.