A new and distinct cultivar of Nemesia plant named 'Sumnem 04', characterized by its upright and compact plant habit; freely branching habit; short internodes; dense and bushy growth habit; early flowering habit; and numerous white-colored flowers.