Abstract: When modelling a natural language, there is already a number of predefined (but not necessarily static) rules for how to use that language, such as its grammar. Deep learning methods have been shown successful when applied to various language modelling tasks. At the same time, applying deep learning gives us result-producing black boxes with no possibility of providing a reasoning for each of the results. Rule-based language models are transparent in that we can either output the rule sequence used as part of the result, or apply the rules in a backwards fashion to explain the result. They also have the advantage that they can make use of information that is already available to us. In this talk, we look at semantic modelling in particular, and explore some of the challenges in rule-based semantic modelling.