• About
  • Advertise
  • Contact
Saturday, May 31, 2025
Manhattan Tribune
  • Home
  • World
  • International
  • Wall Street
  • Business
  • Health
No Result
View All Result
  • Home
  • World
  • International
  • Wall Street
  • Business
  • Health
No Result
View All Result
Manhattan Tribune
No Result
View All Result
Home Science

AI researchers present LLM capable of generating text outputs of up to 10,000 words

manhattantribune.com by manhattantribune.com
16 August 2024
in Science
0
AI researchers present LLM capable of generating text outputs of up to 10,000 words
0
SHARES
0
VIEWS
Share on FacebookShare on Twitter


Since existing LLMs fail to generate sufficiently long output, AgentWrite adopts a plan-then-write pipeline to achieve sufficiently long output with ready-to-use LLMs. Credit: arXiv (2024). DOI: 10.48550/arxiv.2408.07055

A team of artificial intelligence researchers from Tsinghua University, working with a colleague from Zhipu AI, has developed a long language model (LLM) called LongWriter that they claim is capable of generating text output of up to 10,000 words. The group has written a paper describing their efforts and their new LLM, which is available on the website arXiv preprint server.

As LLMs have become commonplace, many have noticed that they are not able to produce very long answers, such as entire books or manuscripts – the current limit seems to be around 2,000 words. The researchers suggest that this is because they are all trained on short documents. In their new initiative, they found that if LLMs are slightly modified and then trained using much longer documents, they are able to produce longer documents.

To test their idea, the research teams first trained a 9-billion-parameter LLM using a conventional dataset, which included documents that were mostly less than 2,000 words long. As expected, when queried, the LLM was unable to create texts longer than 2,000 words.

The team then modified a traditional LLM using a pipeline called AgentWrite to break down the training material into subtasks as it was processed. They then assembled a dataset called “LongWriter-6k,” which contains 6,000 written documents ranging in length from 2,000 to 32,000 words. They then trained the modified LLM using the new LongWriter-6k dataset and found that this increased the length of documents it could produce to around 10,000 words.







Credit: Yushi Bai et al.

Looking at the new long documents produced by the LLM, the team found that they were consistent and usable in a variety of contexts. They have released the open-source code for their model on GitHub, a move that will allow others to draw inspiration from what the team in China has done. They have also released a video showing LongWriter producing a 10,000-word travel guide for people traveling to China.

Researchers acknowledge that there are ethical considerations that need to be taken into account now that it has been discovered that LLMs can generate entire research papers, books, manuscripts or perhaps even film scripts.

More information:
Yushi Bai et al, LongWriter: Unlocking 10,000+ Words Generated from Long-Context LLMs, arXiv (2024). DOI: 10.48550/arxiv.2408.07055

Github: github.com/THUDM/LongWriter

Journal information:
arXiv

© 2024 Science X Network

Quote:AI researchers present LLM capable of generating text outputs of up to 10,000 words (2024, August 16) retrieved August 16, 2024 from

This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without written permission. The content is provided for informational purposes only.



Tags: capablegeneratingLLMoutputspresentResearcherstextwords
Previous Post

Why isn’t Colorado snow flowing into the Colorado River? Research suggests it could be due to lack of spring precipitation

Next Post

Lipreading activates brain regions similar to real speech, researchers show

Next Post
Lipreading activates brain regions similar to real speech, researchers show

Lipreading activates brain regions similar to real speech, researchers show

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Category

  • Blog
  • Business
  • Health
  • International
  • National
  • Science
  • Sports
  • Wall Street
  • World
  • About
  • Advertise
  • Contact

© 2023 Manhattan Tribune -By Millennium Press

No Result
View All Result
  • Home
  • International
  • World
  • Business
  • Science
  • National
  • Sports

© 2023 Manhattan Tribune -By Millennium Press