Blog

Using Perplexity.ai's Spaces as a life raft in an age of AI slop

Kye Fox

— 8 min read

It's well-known at this point how poor traditional search engines have gotten. That was before Large Language Model (LLM)-generated slop flooded the field. Which means it's weird to see me talking about a good use case for LLMs if you've only seen and heard about the bad. I'll hit on that in a future post, maybe. For now: the good stuff!

First: the elephant in the room.

"But what about the environmental cost?" you sagely ask.

So here's the thing: Google invented transformers. That's the paper and technology behind LLMs. They've been using this stuff and its predecessor technologies since the day it was founded. All search engines do.

The ethical issues regarding scraping and citation for LLM-based search are the same ones we collectively debated at length when web search engines first emerged and I'm not sure we ever arrived at a good answer. Debates and lawsuits continue over it.

I don't know if LLMs are worse on energy use than other machine learning-based technologies search engines use, but LLMs are in the mix, and they're here to stay. So why not search less?

What if you could drop a couple of words into a box and have all, or most, of your questions on a topic addressed and maybe even fully answered in an outline?

What if you could do this instead of pleading with Google or Bing or DuckDuckGo for 30, 50, 100 queries trying to find that nugget buried under all the slop churned out by SEO dipshits? (not to be confused with garden variety SEO focused on making pages friendly to both humans and robots)

Welcome to the world where you can do that. It's already here! Neat.

You know how every search engine under the sun is putting an "Answers" thing in there? That's them recognizing the problem and trying to solve it with LLMs. Most of them use a model that they make publicly available for others to use, or use one of those models.

Enter: Perplexity.

Perplexity is an LLM-based–"AI-based" if you must–search engine with a focus on applying these popular models to research. It blows ChatGPT out of the water for pretty much every prompt involving looking stuff up and summarizing. At least in my experience. And it would be unfair to compare it to traditional search engines.

LLM models are trained on all that stuff you try to dig through in traditional search engines, plus the good stuff, and more. They go through a process of human and machine-mediated refinement to get rid of the noise. The exact how seems to change monthly, and it's often secret and proprietary, but the gist is they keep getting better.

Cloud-based LLM search things employ something called Retrieval-Augmented Generation, or RAG for short. This lets a system combine current information with a model that might have seen its last data a year or three ago. You can do this with local LLMs, but enabling RAG on local models is more complicated and out of scope for this article. It also introduces some fun security considerations you have to think about and understand!

All the cloud-based LLM things do search. Perplexity takes it a step further with Spaces. Here's a screenshot of my "The Meta" space in action.

The image shows a search query asking, "What kind of nonsense did JK Rowling say lately?" Below the query are various source links, including Reddit, Newsweek, and smaller news outlets. The "Answer" section summarizes Rowling's controversial statements, particularly focusing on her views on trans people and recent social media activity.
Do you really want to spend more time than that on someone like this?

You basically have it start with a premise about what a space is for. That way you can just drop a link or a phrase and have it do the thing you want it to do without prompting. All the threads produced by this prompt are in the space. Follow-ups are limited on the free plan, but it usually gets everything right on the first try. If it doesn't, that's usually a sign the space's prompt needs work!

Thread Summaries prompt: "Always summarize the linked discussion thread with credited quotes reflecting the different perspectives. Attach the permalink from the quoted comment to the quote in the summary. Include a list of any links with the name of the commentor who shared them."

I've tried to just stop reading Reddit, Hacker News, etc but I do occasionally find something useful there. This space grinds it all down and pans the gold for me. The links to posts don't currently work, but it does provide the user names. I can page search those on Hacker News, and it's usually in the latest comments on Reddit user pages. I have feedback in to Perplexity, so this should work better in the future.

The Gist prompt: "Always respond to links with a summary and outline."

Some people hate spoilers. I don't. And with non-fiction, I find knowing what I'm about to read helps with recall and understanding. I only do this for news and educational type stuff, not personal essays.

The Meta prompt: "Topics I ask about here are something I've heard about but can't find context on. Usually current events. Look up the topic and see what the latest news and discussion is about. Add context if it's something ongoing."

The Meta is mainly used when someone vagueposts on social media. I can drop some odious figure's name in and see what kind of nonsense they said that has everyone talking so I can avoid a bunch of stressful scrolling and searching to figure it out. If a recurring pain in the butt hasn't risen to the level of recent news articles, there's probably nothing new or urgent. I can also drop trending topics in here and completely short-circuit any urge to doomscroll.

That's all I have for now, because these three spaces collectively handle everything I've thrown at them. I haven't created a special purpose space for doing research on a topic yet, but that's an obvious next step.

For subscribers, some interesting bits and pieces follow:

  • A summary of how LLMs are developed, with citations, produced by Perplexity.ai
  • A Perplexity.ai summary of this thread discussing AI, demonstrating my Thread Summaries space

Overview of LLM Development

Large Language Models (LLMs) are advanced AI systems that process and generate natural language, revolutionizing various industries with their capabilities. Developing LLMs involves several complex steps, from model selection to deployment. Here's a summary of the key aspects of LLM development:

Steps in LLM Development

  1. Model Selection: Choosing the right LLM is crucial. Factors include cost, performance, and the model's ability to handle specific use cases. It's advisable to select models from different vendors to avoid lock-in and ensure flexibility for future updates[1][9].

  2. Prompt Engineering: This involves customizing the LLM by crafting effective prompts to achieve desired outputs. It requires skill and subtlety, as the quality of prompts significantly impacts the model's performance[1][9].

  3. Data Curation: Preparing and curating data is essential for training and fine-tuning LLMs. This includes collecting relevant examples of inputs and desired outputs, addressing data privacy issues, and ensuring data quality[9].

  4. Training and Fine-Tuning: Fine-tuning involves adjusting the model's parameters using labeled data specific to the target task. Techniques like LoRA (Low-Rank Adaptation) can make this process more efficient[1][9].

  5. Retrieval-Augmented Generation (RAG): This method enhances LLMs by incorporating external data not included in the initial training. It involves retrieving relevant information, augmenting the prompt, and generating output based on the enriched context[1][10].

  6. Agents and Integration: Agents expand on LLM capabilities by integrating them with other tools and services. Frameworks like LangChain simplify building RAG pipelines and agents, helping to specialize LLMs for specific domains[1][4].

Challenges in LLM Deployment

  • Privacy, Security, and Compliance: Deploying LLMs in enterprise settings raises significant concerns about data security and compliance[2].
  • Technical Challenges: Successfully transitioning LLM applications from prototypes to production is challenging, with high failure rates reported[2].
  • Supply Chain Risks: LLMs can "hallucinate" non-existent packages, posing risks to software supply chains[8].

Current State and Future Directions

  • Advancements: Models like Meta's Llama and Google's Realm are pushing the boundaries of efficiency and accuracy in LLMs[10][11].
  • Applications: LLMs are being used in complex tasks such as code migrations and content generation, with ongoing research into optimizing their performance and reducing training times[3][5].

For more detailed insights into LLM development, consider exploring resources like AI21 Labs' guide to LLM product development and InfoWorld's articles on generative AI development processes[1][9].

Citations:
[1] https://www.infoworld.com/article/2337244/understanding-the-generative-ai-development-process.html
[2] https://www.infoworld.com/article/3540319/how-to-get-llm-driven-applications-into-production.html
[3] https://www.infoworld.com/article/3804552/heres-how-google-is-using-llms-for-complex-internal-code-migrations.html
[4] https://www.infoworld.com/article/2334784/what-is-langchain-easier-development-of-llm-applcations.html
[5] https://www.computerworld.com/article/1627101/what-are-large-language-models-and-how-are-they-used-in-generative-ai.html
[6] https://www.infoworld.com/article/3617664/surveying-the-llm-application-framework-landscape.html
[7] https://www.infoworld.com/article/2336209/is-creating-an-in-house-llm-right-for-your-organization.html
[8] https://www.infoworld.com/article/3542884/large-language-models-hallucinating-non-existent-developer-packages-could-fuel-supply-chain-attacks.html
[9] https://www.ai21.com/blog/llm-product-development
[10] https://www.techtarget.com/whatis/definition/large-language-model-LLM
[11] https://hatchworks.com/blog/gen-ai/large-language-models-guide/
[12] https://www.turing.com/resources/the-complete-guide-to-llm-development
[13] https://aws.amazon.com/what-is/large-language-model/
[14] https://www.cloudflare.com/learning/ai/what-is-large-language-model/
[15] https://developers.google.com/machine-learning/resources/intro-llms
[16] https://towardsdatascience.com/all-you-need-to-know-to-develop-using-large-language-models-5c45708156bc?gi=ccbb88e9a0b1

Answer from Perplexity: pplx.ai/share

The discussion on Hacker News revolves around various topics related to AI models, including the cost and performance of different models like Claude and OpenAI's o3-mini, as well as broader themes about AI's impact on information consumption and interaction.

Cost and Performance of AI Models

  • Cost Efficiency: A user, simonw, noted that using a specific script for Claude cost 3.3612 cents for 18,936 input and 2,905 output. This highlights the cost-effectiveness of certain AI models for processing large amounts of data.

  • Model Comparison: sdesol compared the performance of o3-mini with other models like GPT-4o and DeepSeek R1, noting that o3-mini excelled in handling large context analysis and focusing on developer experience.

Impact of AI on Information Consumption

  • Information Overload: There's a concern about how AI-generated responses can bury relevant information. bredren mentioned that users must navigate through AI responses to find relevant reviews, which can be cumbersome.

  • Context Exhaustion: sdesol noted that current context lengths are insufficient for true general intelligence, suggesting that infinite context length is necessary for universal computation.

AI-Generated Content and Interaction

  • Synthetic Content: genewitch discussed the concept of AI ingesting AI-generated content, suggesting it could lead to new forms of knowledge synthesis.

  • Human Interaction: NiloCK highlighted the importance of peer interaction and reputation building in online discussions, which might be diminished if AI outputs are not integrated into the discourse.

Uncommon Opinions

  • Alternative Formats: acka suggested using Strict YAML as a format for presenting conversations to LLMs, citing its ease of use and compatibility.

  • Mathematical Poetry: DonHopkins shared a poem that uses mathematical concepts to express love, demonstrating a unique intersection of art and science.

Links Shared:

Citations:
[1] https://news.ycombinator.com/item?id=42890627

Answer from Perplexity: pplx.ai/share

Read more