Collie Benchmark

What is Collie?

COLLIE is a grammar-based framework that allows the specification of rich, compositional constraints with diverse generation levels (word, sentence, paragraph…

Released

2023-07-17

Evaluates

强推理, Strong Reasoning, 大语言模型, LLM, 逻辑推理, Reasoning, 不支持, Unsupported

Openness

unknown

Importer

OpenCompass Hub

Reported scores

0

Source provenance

• Original evidence https://hub.opencompass.org.cn/dataset-detail/Collie

Collie paper and code

• Paper https://arxiv.org/abs/2307.08689.
• Code https://github.com/princeton-nlp/Collie

Which sources cite Collie?

• Collie

Related 强推理, Strong Reasoning, 大语言模型, LLM, 逻辑推理, Reasoning, 不支持, Unsupported benchmarks

• A-Bench
• A-OKVQA
• AbsPyramid
• ActionAtlas
• ActiveFly-Bench
• Ada-LEval
• AdvancedMathBench
• AECBench