Simple Summary The milk production traits of goats are economically important. In the Czech Republic, goat milk is processed directly on farms and distributed as cheese. Although goat breeding is not a main focus of animal production in the Czech Republic, it is essential for the agricultural sector. A group of 14 SNPs (single-nucleotide polymorphisms) within four candidate genes (ACACA, BTN1A1, LPL, and SCD) were analysed in two Czech dairy goat breeds, White Shorthaired (WSH) goats and Brown Shorthaired (BSH) goats. The SNPs were significantly associated with milk-production traits (daily milk yield, protein, and fat percentage) and somatic cell count. This information may be useful for marker-assisted selection or related techniques to increase the accuracy of selection. Milk production is influenced by many factors, including genetic and environmental factors and their interactions. Animal health, especially udder health, is usually evaluated by the number of somatic cells. The present study described the effect of polymorphisms in the ACACA, BTN1A1, LPL, and SCD genes on the daily milk yield, fat, and protein percentages and somatic cell count. In this study, 590 White Shorthaired (WSH) and Brown Shorthaired (BSH) goats were included. SNP genotyping was performed by PCR-RFLP and multiplex PCR followed by SNaPshot minisequencing analysis. The linear mixed model with repeated measurement was used to identify the genetic associations between the studied genes/SNPs and chosen traits. All selected genes were polymorphic in the tested goat populations and showed significant associations with milk traits. Only BTN1A1 (SNP g.599 A > G) showed a significant association with the somatic cell score. After Bonferroni correction, a significant effect of LPL g.300G > A on daily milk yield and fat percentage, LPL g.185G > T on protein percentage, and LPL G50C, SCD EX3_15G > A, and SCD EX3_68A > G on fat percentage was found. The importance of environmental factors, such as the herd-year effect, month of milking, and lactation order on all milk performance indicators was confirmed.