跳到内容
当前位置:首页>高中教材>让大模型学会读教材:强化学习怎样奖励自学

让大模型学会读教材:强化学习怎样奖励自学

  • 2026-11-18 03:12:58
让大模型学会读教材:强化学习怎样奖励自学

英文标题:Incentivizing Self-Learning in Large Language Models via Reinforcement Learning

论文日期:2026-10-29

作者:Lingkai Kong、Yuzhe Gu、Haiteng Zhao、Zijian Wu、Youqing Fang、Zhouqi Hua、Yanan Sun、Kai Chen、Wenwei Zhang

来源:arXiv:2610.39671v1

影响因子:不适用(预印本;无期刊JIF年份)

状态:预印本,未经同行评审;官方正文方法及实验关键部分核验

研究解决了什么问题

面对陌生问题,检索到一段材料并不等于学会应用其中原理。如何训练模型在解题途中主动导航、阅读教材,再把新知识用于证明?

方法

BookNavigator提供目录、关键词检索、整节阅读;习题及其答案从可见教材中剥离。基于Intern-S2-Preview-35B作GRPO训练,用最终证明奖励、退火探索奖励及逐轮工具信用分配,学习何时读、读什么、如何用。测试时不更新权重。

主要发现

158本教材构建5183题;数学测试共626题,按每题四次采样的平均正确率计分。同一自学协议由训练前51.5%到79.5%,增28.0个百分点;训练后无书基线68.5%、固定检索最佳73.1%。这是LLM裁判判断的证明正确率,不是形式化证明通过率。跨任务BrowseComp-Plus、CL-bench仅提升1.7和2.3个百分点,CL-bench程序执行子类反降0.6个百分点。

对研究者的实际价值

编辑解读:对科研智能体,值得研究的不只是检索命中率,而是材料是否真正解决了推理中的缺口。课题组可先用方法文档和统计教材构造小型可判分任务,检验阅读策略是否迁移到新分析流程;本文没有直接验证生信或空间转录组。

局限性

证据主要来自一个35B初始模型、教材配对数学任务和两个外部基准。GLM-5.2裁判用八次独立审查,全部通过才判正确,仍不等于独立形式验证。留出教材或学科不保证基础模型预训练从未见过内容。交互阅读最多八轮、64k轨迹,成本和协议与单次固定检索不同;79.5%不能解读为28个百分点全来自检索。未核验到作者明确发布的本论文代码仓库,未运行复现。

研究切入点(编辑建议)

编辑建议:把导航、阅读、应用分别标注,采用人工或可执行检查复核裁判;按教材、方法和时间留出任务。增加同等工具预算与同等token预算对照,比较只奖答案、只奖调用及过程奖励,报告无效调用、延迟和失败类型;重点检验程序执行可靠性是否受损。

原文:

https://arxiv.org/abs/2610.39671

原始 PDF:

https://arxiv.org/pdf/2610.39671

代码:未找到作者明确提供的公开链接。

信息依据:官方arXiv摘要、HTML正文2–3节表1–4、3.1测试拆分和裁判协议,4节测试时权重边界已核验。标题作者截图来自官方摘要页;未找到作者明确提供的公开实现链接。

AI 辅助整理。研究切入点为编辑建议。预印本结论未经同行评审;期刊影响因子不代表单篇论文质量。

基本 文件 流程 错误 SQL 调试
  1. 请求信息 : 2026-11-23 07:52:59 HTTP/1.1 GET : http://c.sjds.net/a/484251.html
  2. 运行时间 : 0.075537s [ 吞吐率:13.24req/s ] 内存消耗:4,733.42kb 文件加载:140
  3. 缓存信息 : 0 reads,0 writes
  4. 会话信息 : SESSION_ID=ffc39e9f7331a8573c7c7b4525a1312d
  1. /www/wwwroot/c.sjds.net/public/index.php ( 0.79 KB )
  2. /www/wwwroot/c.sjds.net/vendor/autoload.php ( 0.17 KB )
  3. /www/wwwroot/c.sjds.net/vendor/composer/autoload_real.php ( 2.49 KB )
  4. /www/wwwroot/c.sjds.net/vendor/composer/platform_check.php ( 0.90 KB )
  5. /www/wwwroot/c.sjds.net/vendor/composer/ClassLoader.php ( 14.03 KB )
  6. /www/wwwroot/c.sjds.net/vendor/composer/autoload_static.php ( 4.90 KB )
  7. /www/wwwroot/c.sjds.net/vendor/topthink/think-helper/src/helper.php ( 8.34 KB )
  8. /www/wwwroot/c.sjds.net/vendor/topthink/think-validate/src/helper.php ( 2.19 KB )
  9. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/helper.php ( 1.47 KB )
  10. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/stubs/load_stubs.php ( 0.16 KB )
  11. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/Exception.php ( 1.69 KB )
  12. /www/wwwroot/c.sjds.net/vendor/topthink/think-container/src/Facade.php ( 2.71 KB )
  13. /www/wwwroot/c.sjds.net/vendor/symfony/deprecation-contracts/function.php ( 0.99 KB )
  14. /www/wwwroot/c.sjds.net/vendor/symfony/polyfill-mbstring/bootstrap.php ( 8.26 KB )
  15. /www/wwwroot/c.sjds.net/vendor/symfony/polyfill-mbstring/bootstrap80.php ( 9.78 KB )
  16. /www/wwwroot/c.sjds.net/vendor/symfony/var-dumper/Resources/functions/dump.php ( 1.49 KB )
  17. /www/wwwroot/c.sjds.net/vendor/topthink/think-dumper/src/helper.php ( 0.18 KB )
  18. /www/wwwroot/c.sjds.net/vendor/symfony/var-dumper/VarDumper.php ( 4.30 KB )
  19. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/App.php ( 15.30 KB )
  20. /www/wwwroot/c.sjds.net/vendor/topthink/think-container/src/Container.php ( 15.76 KB )
  21. /www/wwwroot/c.sjds.net/vendor/psr/container/src/ContainerInterface.php ( 1.02 KB )
  22. /www/wwwroot/c.sjds.net/app/provider.php ( 0.19 KB )
  23. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/Http.php ( 6.04 KB )
  24. /www/wwwroot/c.sjds.net/vendor/topthink/think-helper/src/helper/Str.php ( 7.29 KB )
  25. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/Env.php ( 4.68 KB )
  26. /www/wwwroot/c.sjds.net/app/common.php ( 0.03 KB )
  27. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/helper.php ( 18.78 KB )
  28. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/Config.php ( 5.54 KB )
  29. /www/wwwroot/c.sjds.net/config/app.php ( 0.95 KB )
  30. /www/wwwroot/c.sjds.net/config/cache.php ( 0.78 KB )
  31. /www/wwwroot/c.sjds.net/config/console.php ( 0.23 KB )
  32. /www/wwwroot/c.sjds.net/config/cookie.php ( 0.56 KB )
  33. /www/wwwroot/c.sjds.net/config/database.php ( 2.48 KB )
  34. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/facade/Env.php ( 1.67 KB )
  35. /www/wwwroot/c.sjds.net/config/filesystem.php ( 0.61 KB )
  36. /www/wwwroot/c.sjds.net/config/lang.php ( 0.91 KB )
  37. /www/wwwroot/c.sjds.net/config/log.php ( 1.35 KB )
  38. /www/wwwroot/c.sjds.net/config/middleware.php ( 0.19 KB )
  39. /www/wwwroot/c.sjds.net/config/route.php ( 1.89 KB )
  40. /www/wwwroot/c.sjds.net/config/session.php ( 0.57 KB )
  41. /www/wwwroot/c.sjds.net/config/trace.php ( 0.34 KB )
  42. /www/wwwroot/c.sjds.net/config/view.php ( 0.82 KB )
  43. /www/wwwroot/c.sjds.net/app/event.php ( 0.25 KB )
  44. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/Event.php ( 7.67 KB )
  45. /www/wwwroot/c.sjds.net/app/service.php ( 0.13 KB )
  46. /www/wwwroot/c.sjds.net/app/AppService.php ( 0.26 KB )
  47. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/Service.php ( 1.64 KB )
  48. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/Lang.php ( 7.35 KB )
  49. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/lang/zh-cn.php ( 13.70 KB )
  50. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/initializer/Error.php ( 3.31 KB )
  51. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/initializer/RegisterService.php ( 1.33 KB )
  52. /www/wwwroot/c.sjds.net/vendor/services.php ( 0.14 KB )
  53. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/service/PaginatorService.php ( 1.52 KB )
  54. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/service/ValidateService.php ( 0.99 KB )
  55. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/service/ModelService.php ( 2.04 KB )
  56. /www/wwwroot/c.sjds.net/vendor/topthink/think-trace/src/Service.php ( 0.77 KB )
  57. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/Middleware.php ( 6.72 KB )
  58. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/initializer/BootService.php ( 0.77 KB )
  59. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/Paginator.php ( 11.86 KB )
  60. /www/wwwroot/c.sjds.net/vendor/topthink/think-validate/src/Validate.php ( 63.20 KB )
  61. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/Model.php ( 23.55 KB )
  62. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/model/concern/Attribute.php ( 21.05 KB )
  63. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/model/concern/AutoWriteData.php ( 4.21 KB )
  64. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/model/concern/Conversion.php ( 6.44 KB )
  65. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/model/concern/DbConnect.php ( 5.16 KB )
  66. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/model/concern/ModelEvent.php ( 2.33 KB )
  67. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/model/concern/RelationShip.php ( 28.29 KB )
  68. /www/wwwroot/c.sjds.net/vendor/topthink/think-helper/src/contract/Arrayable.php ( 0.09 KB )
  69. /www/wwwroot/c.sjds.net/vendor/topthink/think-helper/src/contract/Jsonable.php ( 0.13 KB )
  70. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/model/contract/Modelable.php ( 0.09 KB )
  71. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/Db.php ( 2.88 KB )
  72. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/DbManager.php ( 8.52 KB )
  73. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/Log.php ( 6.28 KB )
  74. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/Manager.php ( 3.92 KB )
  75. /www/wwwroot/c.sjds.net/vendor/psr/log/src/LoggerTrait.php ( 2.69 KB )
  76. /www/wwwroot/c.sjds.net/vendor/psr/log/src/LoggerInterface.php ( 2.71 KB )
  77. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/Cache.php ( 4.92 KB )
  78. /www/wwwroot/c.sjds.net/vendor/psr/simple-cache/src/CacheInterface.php ( 4.71 KB )
  79. /www/wwwroot/c.sjds.net/vendor/topthink/think-helper/src/helper/Arr.php ( 16.63 KB )
  80. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/cache/driver/File.php ( 7.84 KB )
  81. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/cache/Driver.php ( 9.03 KB )
  82. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/contract/CacheHandlerInterface.php ( 1.99 KB )
  83. /www/wwwroot/c.sjds.net/app/Request.php ( 0.09 KB )
  84. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/Request.php ( 55.78 KB )
  85. /www/wwwroot/c.sjds.net/app/middleware.php ( 0.25 KB )
  86. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/Pipeline.php ( 2.61 KB )
  87. /www/wwwroot/c.sjds.net/vendor/topthink/think-trace/src/TraceDebug.php ( 3.40 KB )
  88. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/middleware/SessionInit.php ( 1.94 KB )
  89. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/Session.php ( 1.80 KB )
  90. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/session/driver/File.php ( 6.27 KB )
  91. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/contract/SessionHandlerInterface.php ( 0.87 KB )
  92. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/session/Store.php ( 7.12 KB )
  93. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/Route.php ( 23.73 KB )
  94. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/route/RuleName.php ( 5.75 KB )
  95. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/route/Domain.php ( 2.53 KB )
  96. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/route/RuleGroup.php ( 22.43 KB )
  97. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/route/Rule.php ( 26.95 KB )
  98. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/route/RuleItem.php ( 9.78 KB )
  99. /www/wwwroot/c.sjds.net/route/app.php ( 1.72 KB )
  100. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/facade/Route.php ( 4.70 KB )
  101. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/route/dispatch/Controller.php ( 4.74 KB )
  102. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/route/Dispatch.php ( 10.44 KB )
  103. /www/wwwroot/c.sjds.net/app/controller/Index.php ( 4.81 KB )
  104. /www/wwwroot/c.sjds.net/app/BaseController.php ( 2.05 KB )
  105. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/facade/Db.php ( 0.93 KB )
  106. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/db/connector/Mysql.php ( 5.44 KB )
  107. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/db/PDOConnection.php ( 52.47 KB )
  108. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/db/Connection.php ( 8.39 KB )
  109. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/db/ConnectionInterface.php ( 4.57 KB )
  110. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/db/builder/Mysql.php ( 16.58 KB )
  111. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/db/Builder.php ( 24.06 KB )
  112. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/db/BaseBuilder.php ( 27.50 KB )
  113. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/db/Query.php ( 15.71 KB )
  114. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/db/BaseQuery.php ( 45.13 KB )
  115. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/db/concern/TimeFieldQuery.php ( 7.43 KB )
  116. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/db/concern/AggregateQuery.php ( 3.26 KB )
  117. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/db/concern/ModelRelationQuery.php ( 20.07 KB )
  118. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/db/concern/ParamsBind.php ( 3.66 KB )
  119. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/db/concern/ResultOperation.php ( 7.01 KB )
  120. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/db/concern/WhereQuery.php ( 19.37 KB )
  121. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/db/concern/JoinAndViewQuery.php ( 7.11 KB )
  122. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/db/concern/TableFieldInfo.php ( 2.63 KB )
  123. /www/wwwroot/c.sjds.net/vendor/topthink/think-orm/src/db/concern/Transaction.php ( 2.77 KB )
  124. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/log/driver/File.php ( 5.96 KB )
  125. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/contract/LogHandlerInterface.php ( 0.86 KB )
  126. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/log/Channel.php ( 3.89 KB )
  127. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/event/LogRecord.php ( 1.02 KB )
  128. /www/wwwroot/c.sjds.net/vendor/topthink/think-helper/src/Collection.php ( 16.47 KB )
  129. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/facade/View.php ( 1.70 KB )
  130. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/View.php ( 4.39 KB )
  131. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/Response.php ( 8.81 KB )
  132. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/response/View.php ( 3.29 KB )
  133. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/Cookie.php ( 6.06 KB )
  134. /www/wwwroot/c.sjds.net/vendor/topthink/think-view/src/Think.php ( 8.38 KB )
  135. /www/wwwroot/c.sjds.net/vendor/topthink/framework/src/think/contract/TemplateHandlerInterface.php ( 1.60 KB )
  136. /www/wwwroot/c.sjds.net/vendor/topthink/think-template/src/Template.php ( 46.61 KB )
  137. /www/wwwroot/c.sjds.net/vendor/topthink/think-template/src/template/driver/File.php ( 2.41 KB )
  138. /www/wwwroot/c.sjds.net/vendor/topthink/think-template/src/template/contract/DriverInterface.php ( 0.86 KB )
  139. /www/wwwroot/c.sjds.net/runtime/temp/52e0794259aeabc01163982c3f0015ae.php ( 8.05 KB )
  140. /www/wwwroot/c.sjds.net/vendor/topthink/think-trace/src/Html.php ( 4.42 KB )
  1. CONNECT:[ UseTime:0.000727s ] mysql:host=172.18.0.4;port=3306;dbname=c_sjds;charset=utf8mb4
  2. SHOW FULL COLUMNS FROM `fenlei` [ RunTime:0.001075s ]
  3. SELECT * FROM `fenlei` WHERE `fid` = 0 [ RunTime:0.000468s ]
  4. SELECT * FROM `fenlei` WHERE `fid` = 63 [ RunTime:0.000334s ]
  5. SHOW FULL COLUMNS FROM `set` [ RunTime:0.000525s ]
  6. SELECT * FROM `set` [ RunTime:0.000303s ]
  7. SHOW FULL COLUMNS FROM `article` [ RunTime:0.000579s ]
  8. SELECT * FROM `article` WHERE `id` = 484251 LIMIT 1 [ RunTime:0.000379s ]
  9. UPDATE `article` SET `lasttime` = 1795391580 WHERE `id` = 484251 [ RunTime:0.001485s ]
  10. SELECT * FROM `fenlei` WHERE `id` = 66 LIMIT 1 [ RunTime:0.000309s ]
  11. SELECT * FROM `article` WHERE `id` < 484251 ORDER BY `id` DESC LIMIT 1 [ RunTime:0.000420s ]
  12. SELECT * FROM `article` WHERE `id` > 484251 ORDER BY `id` ASC LIMIT 1 [ RunTime:0.000322s ]
  13. SELECT * FROM `article` WHERE `id` < 484251 ORDER BY `id` DESC LIMIT 10 [ RunTime:0.000528s ]
  14. SELECT * FROM `article` WHERE `id` < 484251 ORDER BY `id` DESC LIMIT 10,10 [ RunTime:0.000920s ]
  15. SELECT * FROM `article` WHERE `id` < 484251 ORDER BY `id` DESC LIMIT 20,10 [ RunTime:0.000779s ]
0.085478s