BestBlogs.dev Weekly Selection Issue #3

The article first points out that the development of large models is rapidly advancing, but discussions mainly focus on pricing and applications, with less emphasis on scenario exploration. The author, through contrasting the United States' lead in foundational and application architectures and the potential for China to replicate its glory days of the internet and mobile internet eras in the application layer, forecasts the changes that large models will bring to industries and enterprises.

The article then categorizes generative AI applications into four main types and discusses the impact of large models on work, distinguishing the concepts of Copilot and Agent, the latter having a higher degree of autonomy. It introduces the Agent architecture's "four-piece set" (perception, memory, tools, action), and how this architecture endows Large Language Models (LLM) with strategic thinking structures, simulating the human problem-solving process.

The article further elaborates on the importance of "model," "architecture," and "people" in enabling large models to perform effectively, as well as how to efficiently and cost-effectively construct an Agent, including the derivation from business scenarios to Agent capabilities and how to analyze business scenario SOPs to identify needs. Using the example of a car sales scenario, it explains the abstraction from application scenarios to atomized capabilities and how to implement Agent construction through an intelligent entity assembly platform.

Finally, the article discusses how to enhance the effectiveness of large models by providing high-quality "information," emphasizing the importance of using large models within a safe scope, and how to improve problem-solving effectiveness through prompt engineering. It also mentions the advantages of a multi-Agent collaborative model and the importance of engineering chain design when constructing Agents.

Kunlun's Skywork-MoE, a 200 billion parameter sparse model, is the first to support inference on a single 4090 server, significantly reducing costs. It leverages MoE Upcycling technology, enhancing performance while maintaining a smaller parameter size compared to competitors. The model is fully open-source, including weights, technical reports, and inference code optimized for 8x4090 servers.

Kuaishou's 'Keling' AI video generation model, which is similar to Sora, has opened beta testing. It can generate videos over 2 minutes long, with a resolution of 1080p and a frame rate of 30fps. The model is capable of simulating physical world characteristics and accurately modeling complex motions. It has been integrated into the Kuaishou ecosystem and is available for testing in the Kuaishou app.

As large language models (LLM) continue to grow in scale, their performance has been improving. However, a key challenge remains: aligning with human values and intentions. One powerful technique for this is Reinforcement Learning from Human Feedback (RLHF). With the increase in model size, RLHF typically requires maintaining multiple models and increasingly complex learning processes, leading to higher demands on memory and computational resources. OpenRLHF, an open-source RLHF framework, was proposed by a joint team including OpenLLMAI, ByteDance, NetEase Fuxi AI Lab, and Alibaba. It offers an easy-to-use, scalable, and high-performance solution for RLHF training of models over 70 billion parameters, integrating PPO and other techniques.

This article introduces the concept of prompt injection attacks, detailing how they work, common types, and potential risks. It explains how prompt injection can lead to systems generating incorrect information, writing malware, and even data breaches and remote system takeover. The article also discusses various countermeasures such as data vetting, the principle of least privilege, and reinforcement learning with human feedback.

The article explores the evolution of search engines from keyword-based to hybrid search methods, emphasizing the importance of understanding context in search queries. It introduces a hybrid search engine that combines keyword and semantic search techniques to improve search results. The implementation leverages Cohere for semantic search and pgvector for keyword search within a PostgreSQL database hosted on Timescale Cloud. The article details the architecture, setup, and implementation steps, including embedding generation, storage, retrieval, and reranking. It also discusses the application of this hybrid search engine in a Retrieval-Augmented Generation (RAG) system, demonstrating how to integrate it with LangChain for advanced question-answering capabilities. The article concludes with a practical example using the CNN-DailyMail dataset, showcasing the effectiveness of the hybrid search approach.

This article explores in detail how the Shoumao technology team combines large language models (LLM) with AI Agent technology, addressing the problems encountered, thought strategies, and practical cases throughout the process. The article first introduces the concept of an AI Agent, defined as Agent = LLM + memory + planning skills + tool usage, emphasizing that an Agent needs to have the ability to perceive the environment, make decisions, and take appropriate actions. Next, the article elaborates on the decision-making process of an Agent, which includes three steps: perception, planning, and action, illustrating the execution process of an Agent through specific cases. In an LLM-driven Agent system, the LLM acts as the brain, supplemented by key components such as planning, memory, and tool usage.

Over the past year, the Shoumao team has started to focus on AI technology trends, exploring the combination of Agent technology and shopping group hand business. The article provides a detailed account of the technical challenges, ideas, and practices encountered by Shoumao in integrating Agent capabilities with intelligent assistant services. It presents the end-display solution, Agent abstraction and management, and the construction of an Agent laboratory. In addition, the article discusses the classification, definition, and exception handling of tools, as well as the concept and pros and cons of tool granularity, summarizing the considerations for ensuring tool security.

During the project's launch and iteration process, the Shoumao team encountered several issues, including high requirements for result accuracy, structural display error rates when the large model outputs directly to the end, instability in the Agent's understanding of tools, and the complexity requirements for tool returns by the LLM.

The Hugging Face blog has released a leaderboard for AI image models based on human preferences, aiming to assess and compare the performance of various models. This leaderboard ranks mainstream models, including Midjourney, DALL·E, and Stable Diffusion, among others, based on over 45,000 preference choices. Users can contribute to the rankings by participating in the "Text-to-Image Arena," and after voting on 30 images, they can receive a personalized model ranking.

The leaderboard uses the ELO scoring system, which calculates scores for each model by surveying human preferences across more than 700 images. These images cover a variety of styles and categories to ensure comprehensive evaluation.

Early analysis indicates that proprietary models such as Midjourney and DALL·E 3 HD are currently leading the field. However, open-source models, particularly Playground AI v2.5, are rapidly improving and have surpassed proprietary models in certain areas. Additionally, the upcoming open-source release of Stable Diffusion 3 Medium is expected to have a significant impact on the open-source community.

In this article, the authors delve into the valuable experiences and practical insights gained from building and managing large language model (LLM) applications. The article covers multiple aspects from an operational perspective, including data handling, model management, product design, and team building.

First, the article emphasizes the importance of data quality. Regularly reviewing the discrepancies between development and production environments ensures that the data samples in the development environment are consistent with those in the production environment, helping to prevent performance issues in the actual application. Additionally, the article suggests examining LLM input and output samples daily to quickly identify and adapt to new patterns or failure modes.

In terms of model management, the authors recommend generating structured outputs to simplify downstream integration and discuss the challenges of migrating prompts between different models. To ensure stable model performance, the authors advise using version control and fixing model versions to avoid unexpected changes due to model updates. Moreover, selecting the smallest model that can accomplish the task can effectively reduce latency and cost.

For product design, the article points out that designers should be involved early and frequently in the development process. This involvement should go beyond merely enhancing the interface; designers should rethink the user experience and propose valuable improvements. Designing human-in-the-loop user experiences, allowing users to provide feedback and corrections, can enhance the immediate output quality of the product and collect valuable data for model improvement. Clarifying the prioritization of requirements and adjusting risk tolerance according to use cases are also crucial for success.

In team building, the article stresses the importance of focusing on processes rather than merely relying on tools. Cultivating a culture of experimentation and encouraging the team to conduct experiments and iterations can help discover the best solutions. Ensuring that all team members can access and utilize the latest AI technologies and recognizing that a successful LLM application team requires a diverse set of skills, including data science, software engineering, and product design, are also highlighted.

Overall, this article provides profound insights into effectively developing and managing LLM applications and serves as a practical operational guide for professionals in the field.

This article discusses how ZhuiPu AI has launched its new GLM-4 series and other products, significantly reducing the cost of large model usage. With enterprise discounts, the GLM-4-Flash model can be used for less than one cent to generate content equivalent to two copies of the classic novel 'Dream of the Red Chamber.' ZhuiPu AI's open platform now serves 300,000 enterprise clients, with a daily token usage reaching 40 billion. The article highlights the advancements in their MaaS 2.0 platform, the release of an open-source 9B parameter model, and the increased capabilities of AI agents in various applications.

In recent studies, researchers from Meta, Paris-Saclay University, and Paris-Sorbonne University have jointly proposed a new training method for large language models. This method improves the sample efficiency of the models by predicting multiple future tokens simultaneously, rather than the traditional single token prediction. The new approach has shown advantages in both code generation and natural language generation tasks, without increasing training time, and can even triple the inference speed. Experiments indicate that as the size of the models increases, the benefits of this method become even more pronounced, especially during training with multiple epochs. In benchmark tests for generative tasks such as programming, the performance improvement of models trained with multi-token prediction is particularly significant.

The article discusses the importance of using Retrieval-Augmented Generation (RAG) systems in LLM applications to enhance the accuracy and reliability of LLM responses by vectorizing data. RAG systems enable LLM to retrieve and reference specific data within the semantic space by chunking and converting data into vectors. The size of these data chunks is crucial for the accuracy of search results; chunks that are too large can lack specificity, while those that are too small may lose context. The article also cites insights from Roie Schwaber-Cohen of Pinecone, emphasizing the role of metadata in filtering and linking to original content, as well as how different chunking strategies can affect the efficiency and accuracy of the system.

Several common chunking strategies are outlined, including fixed-size chunking, random-size chunking, sliding window chunking, context-aware chunking, and adaptive chunking. Each strategy has its advantages and limitations, and the most suitable method must be chosen based on the specific use case. For instance, Stack Overflow implemented semantic search by treating questions, answers, and comments as discrete semantic chunks according to the structure of the page. Ultimately, determining the optimal chunking strategy involves actual testing and evaluation to optimize the performance of the RAG system.

This article discusses OpenAI's latest research on understanding the 'thinking' of GPT-4. The study introduces sparse autoencoders as a method to identify key points within AI models, enabling better utilization. OpenAI has developed a new approach allowing sparse autoencoders to be extended to millions of features, outperforming previous methods. The article includes a paper, code repository, and an interactive viewer for exploring the concept further.

Since its introduction in 2017, Transformer has become the mainstream architecture for AI large models, especially in language modeling. However, its limitations have become apparent with the expansion of model size and sequence length. The Mamba model, introduced a few months ago, addressed some of these issues by achieving linear scalability with context length. Now, the original authors have released Mamba 2, which offers significant improvements in training efficiency and performance. Key contributions include the development of the SSD (state space duality) framework, improved linear attention theory, and the introduction of new algorithms that leverage larger state dimensions. Mamba 2 outperforms its predecessor and other models in various tasks, demonstrating the complementary nature of attention mechanisms and state space models.

The article discusses a new reinforcement learning framework, RL4VLM, which allows multimodal large models to learn decision-making tasks without human feedback. Key points include: 1) The model can perform tasks like playing poker and solving '12 points' problems, surpassing GPT-4v. 2) The framework uses environmental rewards instead of human feedback, enhancing decision-making capabilities. 3) The model's performance was tested on tasks requiring fine-grained visual information and embodied intelligence. 4) The framework integrates visual and textual inputs for task states and uses PPO for fine-tuning.

This podcast features Edo Liberty, Founder and CEO of Pinecone, discussing the significance of vector databases in facilitating the adoption of Generative AI and LLM-based applications, contrasting them with traditional data stores. Key points include the application of vector databases in various fields and the importance of data security and governance.

Zhipu AI announced a series of advancements in its large models at the recent AI Open Day. The company's Large Model Open Platform currently has 3 million registered users and processes an average of 400 billion tokens per day. The growth of the GLM-4 model over the past 4 months has exceeded 90 times, and the new version, GLM-4-9B, has comprehensively surpassed Llama 3 8B. The multimodal model GLM-4V-9B has also been launched, with all large models remaining open source. The MaaS platform has been upgraded to version 2.0, lowering the threshold for applying large models and providing a more streamlined process for deploying private models. Zhipu AI's commercialization strategy has not only achieved a continuous reduction in application costs through technological innovation but also ensured the upgrade of customer value. Zhipu AI has also introduced the GLM-4-AIR model at a lower price, with performance comparable to GLM-4-0116 but at only 1/100th of the cost. Additionally, Zhipu AI has played a role in formulating AI security standards, joining several international companies in signing the Frontier AI Safety Commitment. Zhipu AI believes that with the Scaling Law still effective, 2024 will be the pivotal year for AGI.

Altera is a company dedicated to creating AI gaming companions with human-like characteristics. Their first product is an AI partner that can explore and interact with players in Minecraft. Unlike the earlier Voyager, Altera's AI focuses more on empathy and emotional interaction, aiming to become a long-term companion for players, rather than just a tool or assistant. The founder, Guangyu Robert Yang, and his team have a strong academic background in neural networks and cognitive science, combining deep learning and behavior modeling to drive the development of this innovative AI.

Altera's vision extends beyond the gaming industry. They aim to build a world of multiple agents, allowing these digital humans to play roles in various fields, and even have their own forms in the physical world. With the advancement of AI technology, these human-like digital beings could change the way we interact with the digital world, breaking the boundaries between virtual and real worlds.

Jina AI's new multimodal embedding model, Jina CLIP v1, significantly outperforms OpenAI's original CLIP model in various retrieval tasks. It provides state-of-the-art performance in both text-only and text-image cross-modal retrieval, eliminating the need for separate models for different modalities. Key improvements include:

1. Enhanced performance in text-only and image-to-image retrieval.
2. Support for longer text inputs with an 8k token input window.
3. Utilization of the EVA-02 model for superior image embeddings.
4. Detailed instructions for getting started with Jina CLIP v1 via Embeddings API and Hugging Face.

The AI Gateway framework serves as a solution to the challenges of AI services, ensuring their reliability, scalability, and manageability. This framework consists of three tiers: foundational architecture, core building blocks, and gateway operations. The foundational architecture involves the integration and implementation of the AI Gateway within applications and the management of API traffic through advanced proxy mechanisms. The core building blocks encompass critical functionalities such as logging, request forwarding, tagging API calls and responses, modifying requests and responses, circuit breaker functionality, metrics collection, and token management. The gateway operations tier is concerned with advanced implementations of cost management, reliability, security, and scalability, including controlling prompt size, user-level rate limits, semantic caching, fallbacks for LLM APIs, filtering system responses, preventing the leakage of personally identifiable information, and caching repetitive API calls. The article concludes with a practical example of an AI Gateway operation flow, demonstrating how to restrict the number of API requests a user can make in a real-world application.

NotebookLM is an AI-powered research and writing assistant developed by Google, which has been upgraded to the Gemini 1.5 Pro version and is being promoted globally. The tool aims to help users better understand complex materials, establish connections between pieces of information, and accelerate the drafting process.

Users can upload research notes, interview transcripts, company documents, and other source materials, and NotebookLM instantly becomes an expert on these materials. The latest upgrade includes support for Google Slides and web URLs, as well as inline citation features that directly guide users to supporting paragraphs in the source materials, facilitating the verification of AI-generated content or deeper research into the original text. Additionally, NotebookLM can convert source materials into useful formats such as FAQs, briefing documents, or study guides to provide a high-level understanding of the source materials.

Alibaba's Tongyi Qianwen team has released the Qwen2 series of open-source models, including five sizes: Qwen2-0.5B, Qwen2-1.5B, Qwen2-7B, Qwen2-57B-A14B, and Qwen2-72B. These models are now available on the SiliconCloud platform. Qwen2 models excel in various benchmarks, significantly outperforming leading models like Llama-3-70B and Qwen1.5-110B in natural language understanding, knowledge, code, mathematics, and multilingual capabilities. The Qwen2-72B-Instruct model shows a good balance between fundamental capabilities and alignment with human values. The models also support long-context processing up to 128K tokens and have enhanced multilingual performance. In terms of safety, Qwen2-72B-Instruct is comparable to GPT-4 and significantly better than Mistral-8x22B.

1. Jensen Huang showcased the latest mass-produced version of the Blackwell chip and announced that the Blackwell Ultra AI chip will be launched in 2025. The next-generation AI platform will be named Rubin, with Rubin Ultra expected in 2027. The update cycle will be "once a year," breaking "Moore's Law."

2. Jensen Huang claimed that NVIDIA has driven the birth of large language models. After 2012, NVIDIA changed the GPU architecture and integrated all new technologies into a single computer.

3. NVIDIA's accelerated computing technology has helped achieve a 100-fold increase in speed, with power consumption only tripling and costs increasing by just 1.5 times.

4. Jensen Huang predicts that the next generation of AI will need to understand the physical world. He suggests that AI should learn through videos and synthetic data, and that AIs should learn from each other.

5. In his presentation, Jensen Huang even assigned a Chinese translation to the term "token," calling it "词元" (cí yuán).

6. Jensen Huang stated that the era of robots has arrived, and in the future, all moving objects will achieve autonomous operation.

In the rapidly changing landscape of 2024, many new trends have emerged in the marketing field. This article delves into how these changes impact brand strategies. It first introduces the top 10 marketing trends of 2024, including the rise of VUCA environments, the digital economy, and AI-driven economies, as well as the emergence of a new generation of consumers. The article emphasizes the need for brands to focus on emotional value and the content economy.

The article provides a detailed analysis of three main brand strategies: positioning strategy, audience-centric approach, and co-creation. The positioning strategy aims to capture a unique place in the consumer's mind through category leadership and super symbols but faces limitations in resources and innovative thinking. The audience-centric approach leverages the DTC (Direct-to-Consumer) model, addressing specific group needs, building deep connections with users, and achieving interaction through refined operations. The co-creation strategy emphasizes involving 1% of users in brand creation, jointly creating content, products, and communities to achieve shared growth.

Through rich case studies, the article demonstrates the application of these strategies while also pointing out their limitations. The positioning strategy might be difficult to implement due to resource constraints, the audience-centric approach requires a deep understanding of digitalization, and co-creation needs brands to deeply empower user participation.

In a deep interview with Stripe CEO Patrick Collison, NVIDIA CEO Jensen Huang shared his experiences and management philosophies that led the company to achieve tremendous success. Through this in-depth conversation, Huang's leadership style, innovative thinking, and insights into AI technology were revealed.

Huang emphasized that great achievements require pain and struggle, and not all work continuously brings you joy. He believes that striving and solving difficulties are essential to truly realize the greatness of what you are doing. NVIDIA's management model is also unique, with over 60 executives reporting directly to him, ensuring transparent and efficient information dissemination. This flat management structure not only reduces hierarchy but also promotes internal learning and growth within the company.

In team management, Huang insists on not giving up on any employee easily, believing that everyone has potential. He provides open feedback on mistakes, allowing the entire team to learn and progress. He also stresses that the role of a CEO is to handle tasks that others cannot and to only participate in meetings that drive development and solve problems.

Huang prefers to create entirely new markets rather than compete in existing ones. He believes that innovation and logical reasoning are key to proving the feasibility of ideas. He compares the current AI revolution to the industrial revolution, producing tokens and floating-point numbers, which represent intelligence and will significantly enhance productivity across various industries, with vast potential.

The article also explores significant breakthroughs in the AI field with ChatGPT and Llama. ChatGPT democratized computing, while Llama democratized generative AI, fostering widespread application and research. Huang emphasizes that actively engaging with AI is crucial for future competition; otherwise, you will be replaced by those who utilize AI.

Additionally, Huang believes that excellent operations can create good things, but love and care are needed to create extraordinary things. He thinks that products should be beautiful and elegant, striking a balance between simplicity and complexity to provide an exceptional user experience.

The article consists of a dialogue between Kai Qu and Jinjian Zhang, revolving around the core concept of "vitality." Zhang believes that vitality is the intrinsic energy of every individual, the energy and desire for goodness and connection with all things that one feels upon waking each day. He points out that a person with vitality has two characteristics: pragmatism and unconditional self-love. A pragmatic person can respond objectively to problems, while self-love helps people face setbacks and transform suffering into growth. The article also discusses the relationship between vitality and entrepreneurial success, as well as how to achieve self-love. Zhang emphasizes that the core of self-love is forgiveness, which is the central tenet of all religions.

The article further explores how setbacks can promote personal growth and how entrepreneurs can enhance their vitality through overcoming adversity. Kai Qu shares his perspective, believing that those who have experienced setbacks can better understand management and empathize with others, while those who appear to have had smooth sailing may develop an inflated ego due to a lack of hardship experience.

When discussing the relationship between an entrepreneur's script and investment, Zhang mentions the insight and desire that founders must possess. He argues that insight is a reflection of diversity rather than sophistication, and that entrepreneurs should embrace the diversity of the world. Desire is what connects a person to things they truly want to do, not a motive based on comparison. He also notes that a person's stability stems from faith, which is discovered through a process of noise reduction rather than seeking.

Finally, the article discusses the development and future trends of AI. Zhang believes that AI is accelerating and may significantly reduce the demand for programmers within the next three years. He also points out that the integration of AI and blockchain could usher in a new era, and the simultaneous explosion of these technologies will have profound social impacts. Despite market cycles, Zhang believes that value creators will become fewer while money will increase. Therefore, it is important to focus on one's work and find ways to love oneself, such as pursuing a hobby, to maintain a peaceful mindset and nourish creativity.

The article focuses on the strategic layouts of four major tech companies in the field of artificial intelligence.

Firstly, for Google, its strategy in the AI domain is unique, adopting an integrated approach. Through its proprietary TPU processors and the Vertex AI platform, Google provides AI solutions for both consumers and enterprises. Amazon, on the other hand, offers modular services via its Bedrock managed development platform, emphasizing the importance of data gravity. Microsoft, through its collaboration with OpenAI and its own Azure platform, follows a technological approach to ensure the widespread application of AI technology. Meanwhile, Meta opts to open source its large models, such as Llama, to reduce inference costs. By leveraging an open-source strategy, Meta attracts a vast number of developers and enterprises, thereby increasing the usage and improvement speed of its models.

The article points out that both integration and modularization have their advantages. Integration can offer more optimized and coordinated solutions, whereas modularization provides greater flexibility and adaptability.