An increasing number of manufacturing companies have initiated designing and implementing AI systems in manufacturing, however, with limited success. Within our overarching research objective of establishing a methodology for the development of AI systems in manufacturing with socio-technical system consideration, this paper focuses on the early design phase of the development life cycle and aims to identify factors that are essential in the phase but whose importance has been less addressed in the manufacturing literature. To this aim, a case study was conducted adopting a design science approach. The case company was developing an ML-based anomaly detection system for a casting process. The researcher organised an AI system design workshop where participants from the company used the Human-AI design guidelines created by a leading large software company. The workshop enabled the participants to explore a wide range of design concerns. It, however, caused the confusing experience that they had to deal with too many questions simultaneously without clear guidance. Analysing this negative experience has led to identifying four design issues requiring further attention in the research. An example of these issues is that the interdependency of design decisions on operational procedures, human-machine interfaces, ML models, pre-processing, and input data makes it challenging to design these elements in isolation. The study found that a structured approach to dealing with the identified issues was currently lacking. This paper contributes to the manufacturing research community by addressing key unresolved issues in the research through highlighting practical details of designing AI systems in manufacturing.