Evaluation of Generated Poetry

AbstractWe propose a range of automated metrics for evaluation of generated poetry.The metrics measure various aspects of poetry: rhyming, metre, syntax, semantics, and amount of unknown words.In a case study, we implement the metrics for Czech language, apply them to poetry generated by several automated systems as well as human-written, and correlate them with human judgment.We find that most of the proposed metrics correlate well with corresponding human evaluation, but semantically oriented metrics are much better predictors of the overall impression than metrics evaluating formal properties.