Packages defined with environment.systemPackages in nix-darwin are installed to /var/current-system/sw/bin. Those configured with home-manager go to the user-specific profile’s bin directory, typically found at ~/.nix-profile/bin.

THIS is Why List Comprehension is SO Efficient! - YouTube

Python list comprehension is faster than using the .append() method in a loop because it utilizes a special LIST_APPEND bytecode, which is more efficient than the costly object method lookup.

I have very limited experience about goto statement except that I know it is not encouraged in high level language. Today I learn from a comment of why is it illegal to use “goto” that goto statement are commonly used in Linux kernel in resource acquisition/release pattern.

The goto statement comes in handy when a function exits from multiple locations and some common work such as cleanup has to be done. If there is no cleanup needed then just return directly.

Centralized exiting of functions - Linux kernel coding style

Context: Resource Acquisition and Release

• In C, managing resources like memory, file handles, or network connections is critical. These resources must be acquired before use and released after use.
• The process of acquiring and releasing resources is prone to errors. If not handled correctly, it can lead to issues like memory leaks, where memory isn’t freed correctly, or resource exhaustion, where resources aren’t released back to the system.

Why goto is Useful Here

Simplifies Error Handling: When acquiring multiple resources, each step might fail. Normally, you’d need nested if statements to handle these failures. However, with goto, you can jump to the specific code that releases resources, simplifying the error handling.

Clarity and Maintenance: By using goto for forward jumps, the flow of error handling becomes linear and more readable. This is in contrast to nested if statements, which can become complex and difficult to follow.

Example

int function() {
    int *resource1 = acquireResource1();
    if (resource1 == NULL) goto error1;

    int *resource2 = acquireResource2();
    if (resource2 == NULL) goto error2;

    // Use resources...

    releaseResource2(resource2);
    releaseResource1(resource1);
    return SUCCESS;

error2:
    releaseResource1(resource1);
error1:
    return FAILURE;
}

In this example, if acquiring resource2 fails, the code jumps to error2, where resource1 is released. This ensures that even if there is an error in acquiring the second resource, the first one is properly released

This video gives an easy explanation about RAG(retrieval augmented generation)

https://youtu.be/T-D1OfcDW1M?si=a_06Z9VkMSwuh6To

Here is my notes about RAG

limitations of Large Language Model (LLM)

• Outdated Information: LLMs are often limited by the data they were trained on, which can become outdated over time. This leads to responses that may no longer be accurate or relevant.
• Lack of Fact-Checking: Traditional LLMs do not have a mechanism to fact-check or verify the information they generate, which can lead to inaccuracies in their responses.

why was RAG invented

The RAG technique was invented to enhance the accuracy and relevance of responses generated by LLMs. The key motivations for developing RAG were:

• To Provide Up-to-Date Information: By integrating a retrieval system, RAG ensures that the LLM has access to the most current information, thereby improving the accuracy of its responses.
• To Improve Fact-Checking Capabilities: RAG allows LLMs to cross-reference information with reliable sources, enhancing their ability to verify facts and provide more trustworthy responses.
• To Address the Static Nature of LLMs: Since LLMs are trained on static datasets, they can become outdated. RAG introduces a dynamic element where the model can access and incorporate new and updated information.

how does it work

The RAG framework works by combining the capabilities of a traditional LLM with an external information retrieval system. The process involves:

• Retrieval of Relevant Information: When a query is made, the RAG system first retrieves relevant information from an external data store. This data store is continuously updated with new information.
• Three-Part Prompt Processing: The LLM receives a three-part prompt consisting of the instruction, the retrieved content, and the user’s question.
• Generation of Informed Responses: The LLM uses both the retrieved information and its pre-existing knowledge base to generate a response. This ensures that the response is not only based on its training data but also on the most current information available.
• Continuous Updating: The data store used for retrieval is continuously updated, allowing the LLM to stay current with new information and developments.

example of the three-part prompt

• Instruction: This is a directive or guideline that tells the language model what kind of response is expected. It sets the context or the goal for the response.
• Retrieved Content: This part includes the relevant information retrieved from a large-scale data store. This content is dynamically sourced based on the query, ensuring that the information is up-to-date and pertinent to the user’s question.
• User's Question: This is the actual query or question posed by the user. It’s what the user wants to know or the problem they need solved.

Example of a Three-Part Prompt

Here’s how the three-part prompt might looks like:

Use the following pieces of context to answer the question at the end.
If you don't know the answer, just say that you don't know, don't try to make up an answer.
Use three sentences maximum and keep the answer as concise as possible.
Always say "thanks for asking!" at the end of the answer.

<An article or a set of articles from a reputable source, retrieved by the system, discussing the latest developments in AI as of 2023.>

Question: "What are the latest advancements in artificial intelligence?"

refer langchain

A nice way to add arbitrary HTML tags in Hugo site markdown content.

Create a content.html in layouts/partials

{{
.
| replaceRE `\{\^([^}]*)\}` "<sup>$1</sup>"
| replaceRE `\{\_([^}]*)\}` "<sub>$1</sub>"
| replaceRE `\{\#([^}]*)\}` "<kbd>$1</kbd>"
| replaceRE `\{\!([^}]*)\}` "<mark>$1</mark>"
| replaceRE `\{\=([^}]*)\}` "<cite>$1</cite>"
| replaceRE `\{\+([^}]*)\}` "<ins>$1</ins>"
| replaceRE `\{\$([^}]*)\}` "<var>$1</var>"
| safeHTML 
}}

and use the partial html template {{ partial "content.html" .Content }} to render the HTML

Get missing inline HTML tags in Markdown without enabling HTML - tips & tricks - HUGO

display

System Settings > Accessibility > Display > Turn On Reduce Motion

modifier key

• ⌘ change to ^
• caps lock change to ⌘

finder

# show hidden files
defaults write com.apple.finder AppleShowAllFiles YES

# show path bar
defaults write com.apple.finder ShowPathbar -bool true

killall Finder;

brew

install brew

Quick Look plugins

brew install --cask \
    qlcolorcode \
    qlstephen \
    qlmarkdown \
    quicklook-json \
    qlprettypatch \
    quicklook-csv \
    betterzip \
    webpquicklook \
    suspicious-package

applications

brew install --cask \
    appcleaner \
    google-chrome \
    iterm2 \    
    raycast \
    visual-studio-code

command line tools

# [Nerd Font](https://www.nerdfonts.com/)
brew install --cask font-fira-code-nerd-font

brew install \
    bat \
    fish \
    git \
    git-delta \
    go \
    hugo \
    jq \
    neofetch \
    orbstack \
    ripgrep \
    starship \
    tree \
    wget

git

git config --global alias.ci commit
git config --global alias.co checkout
git config --global alias.ss status

git-delta

~/.gitconfig

[core]
    pager = delta

[interactive]
    diffFilter = delta --color-only

[delta]
    navigate = true    # use n and N to move between diff sections
    light = false      # set to true if you're in a terminal w/ a light background color (e.g. the default macOS terminal)

[merge]
    conflictstyle = diff3

[diff]
    colorMoved = default

fish shell

starship preset nerd-font-symbols -o ~/.config/starship.toml

add to ~/.config/fish/config.fish

# Add HomeBrew's bin directory to path so you can use HomeBrew's binaries like `starship`
# Fish uses `fish_add_path` instead of `export PATH` modify $PATH.
fish_add_path "/opt/homebrew/bin/"
# Enable Starship prompt
starship init fish | source

Package manager

fisher

plugin:

• z

abbreviation

add to ~/.config/fish/config.fish

source ~/.config/fish/abbreviation.fish

create abbreviation.fish

abbr proxyall "set --export http_proxy http://127.0.0.1:7890; set --export https_proxy http://127.0.0.1:7890"

custom function

it’s very easy to add a custom function in fish shell, an example

Visual Studio Code

Settings

Side Bar:Location change to right

extensions

• Auto Hide
• AutoTrim
• Emacs Friendly Keymap
• Indenticator
• Sort lines
• vscode-icons

Python

use pyenv to manage Python environments, don’t reply on the python installed by brew, because it might update Python version upexpecetdly when performs brew update.

brew install readline xz pyenv
# otpinal: setup pyenv with fish shell
alias brew="env PATH=(string replace (pyenv root)/shims '' \"\$PATH\") brew"
exec "$SHELL"

pyenv install 3.11.6
pyenv global 3.11.6

Reference

• https://opensource.com/article/19/5/python-3-default-mac

Terminal emulator

iTerm2

Keyboard setting

Use fn to Change Input Source

Keyboard Shortcuts -> Uncheck all Input Sources Spotlight

Apple Internal Keyboard

Keyboard Shortcuts -> Modifier Keys

• Caps Lock Key -> Command
• Command -> Control

External Keyboard

Keyboard Shortcuts -> keep Modifier Keys unchanged

Karabiner-Elements

• left_command -> left_control
• left_control -> left_command
• right_command -> right_control
• right_option -> fn

Make Caps Lock → Hyper Key (⌃⌥⇧⌘) (Caps Lock if alone) import

Tiling Windows Manager

• yabai (no need to disable System Integrity Protection)
• skhd

yabai installation and configuration reference:

• System Settings
• brew install
• typical config

skhd config

cmd + ctrl + shift + alt is the Hyper Key configurated using Karabiner-Elements

# change focus between external displays (left and right)
cmd + ctrl + shift + alt - p : yabai -m display --focus west
cmd + ctrl + shift + alt - n : yabai -m display --focus east

# fullscreen window
cmd + ctrl + shift + alt - m : yabai -m window --toggle zoom-fullscreen

# cycle focus between windows
cmd + ctrl + shift + alt - o : yabai -m window --focus prev || yabai -m window --focus last
# cycle swap window to the main window
cmd + ctrl + shift + alt - j : /bin/bash ~/bin/cycle_clockwise.sh; yabai -m window --focus prev || yabai -m window --focus last

# rotate layout clockwise
cmd + ctrl + shift + alt - r : yabai -m space --rotate 270
# flip along y-axis
cmd + ctrl + shift + alt - y : yabai -m space --mirror y-axis
# flip along x-axis cmd + ctrl + shift + alt - x : yabai -m space --mirror x-axis
# toggle window float
cmd + ctrl + shift + alt - f : yabai -m window --toggle float --grid 4:4:1:1:2:2

cycle_clockwise.sh:

#!/bin/bash

win=$(yabai -m query --windows --window last | jq '.id')

while : ; do
    yabai -m window $win --swap prev &> /dev/null
    if [[ $? -eq 1 ]]; then
        break
    fi
done

Jotting down thoughts helps clarify the mind. The end goal is simple: just write without fretting over tools. That’s why I’ve always found that starting to write with pen and paper is the easiest approach. However, these days, since everyone carries their mobile phone, the trend has shifted towards writing on phones.

At first, I didn’t consider Apple Notes to be a suitable professional note app, so I tried out OneNote, EverNote, and Notion. Unfortunately, none fit my needs. OneNote posed difficulties on mobile devices; its free layout might be handy for stylus use, but on a small screen, it made things hard to identify and search. I gave Notion a shot for half a year, but it was too complex. While it’s great for team collaboration, I didn’t require that functionality, and I was frustrated by the ’everything as a block’ design, which made selecting and formatting a hassle. Notion’s AI is decent, but I wasn’t keen on shelling out another $10 every month on top of ChatGPT Plus. As for Evernote, I mainly used it to collect webpages rather than for writing.

Apple Notes has seen significant enhancements over time. With the introduction of a transformer model in iOS 17, Apple has elevated its autocorrect and predictive text capabilities , understanding and adapting to your typing patterns. It’s even capable of autofilling entire sentences on occasion. My increasing use of Apple Notes on iPhone has led to more frequent use on macOS as well, and the smooth integration across devices enhances the writing experience in a way that other note apps on different platforms can’t match. The autocorrect and inline predictive text feature available on both operating systems is incredibly effective and is arguably unbeatable at the moment.

Apple Notes now features a tag system and smart lists for note management. Combined with other functions like PDF import, file attachment, OCR, and drawing capabilities, Apple Notes meets most of my requirements.

My current workflow involves using Notion for web clipping and Apple Notes for all my personal notes. This approach has taught me to focus less on the tools and more on simply writing. Just write—use the simplest tool at hand and push it to its limits. Only then consider upgrading your tools if necessary.

中文字体的挑战

与只含几十个字母加上常用符号的拉丁字体不同，一个完整的中文字体字符集超过 1 万个字符。中文常用字有 3500+个，而《通用规范汉字表》 则收录了 8000 多个汉字。因此中文字体的大小远超英文字体，往往从 10MB起步¹。这意味着中文字体大多预装在操作系统内，而不适宜作为网站资源直接加载。

可变字体（ Variable Fonts ）

单一常规字体的体积已经相当可观，如果还需要支持粗体、斜体等不同字形（如Bold Italic），每种字形又得加载独立的字体文件，总字体文件的体积就会成倍增加。因此， variable fonts 技术应运而生，它通过算法将多种字形融合进一个文件中，减少了总体积，尽管总体积会略大于单个字体文件。例如， Adobe 在其 Source Han Sans 字体的介绍中说明了一些技术细节，最终能以 4.2MB 的 woff2 格式提供可变字体。不过，即使如此，在网页中直接引用一个 4.2MB 的字体文件也会显著影响网页加载速度。

解决方案

目前针对网页中使用中文字体的最佳实践是，将中文字体分割成多个包，并利用CSS中的unicode-range特性来实现字符的按需加载。这样，只有在网页上出现需要渲染的中文字时，才会加载对应的资源包。通常情况下，只需加载几个包含常用字符的字体包。

The unicode-range CSS descriptor sets the specific range of characters to be used from a font defined by @font-face and made available for use on the current page. If the page doesn’t use any character in this range, the font is not downloaded; if it uses at least one, the whole font is downloaded.

MDN Web Docs

使用 local google fonts 在线工具可以轻松实现字体拆分和生成 CSS 引用。下载 CSS文件后，请确保修改src: url()以使用正确的路径。

CSS 配置

有了正确的 font-face 和中文字体之后，应该在 CSS 中设置中文字体的 fallback, 使用Noto Sans SC添加比较通用的 fallback 字体设置如下

font-family: 'Noto Sans SC', "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei", "WenQuanYi Micro Hei", sans-serif;

References

• self hosting google web fonts
• 中文字体的终极解决方案——对字体进行切片
• Improve font fallbacks

In Emacs, the fill-paragraph function neatly reflows a block of text, wrapping lines to fit within a specified column width. VS Code offers an extension called “Rewrap,” which you can install to enjoy similar functionality.

• Selecting the text you want to rewrap and pressing Alt+Q, or
• Placing your cursor within a comment block and pressing Alt+Q to rewrap that comment.

This feature can increase your productivity significantly if you’re editing or writing a considerable amount of text. It streamlines the process of formatting paragraphs, ensuring that they look neat and are easier to read, very much like Emacs’ fill-paragraph.

Rewrap - Visual Studio Marketplace

I recently ventured into deploying a service on Google Cloud Run. My goal was straightforward: create a service that fetches webpage titles and captures screenshots of URLs. However, the journey led me into a peculiar bug when I actually used it on Goole Cloud Run.

The Bug

During the development phase, I worked with a python:3.11-slim base image on macOS, and my Dockerfile functioned without a hitch. Here’s a snapshot of the Dockerfile I used:

from python:3.11-slim

RUN apt-get update && \
    apt-get install -y git && \
    python -m pip install --upgrade pip && \
    pip install -r requirements.txt && \
    pip install pytest-playwright && \
    playwright install-deps && \
    playwright install && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

Yet, upon deploying to Google Cloud Run and initiating the screenshot capture process, I hit a snag:

playwright._impl._api_types.Error: Executable doesn't exist at /home/.cache/ms-playwright/chromium-1084/chrome-linux/chrome
╔═════════════════════════════
║ Looks like Playwright was just installed or updated.                   
║ Please run the following command to download new browsers: 
║                                                                                                            
║     playwright install                                                                          
║                                                                                                            
║ <3 Playwright Team                                                                         
╚═════════════════════════════

Official Playwright Docker Image Saves the Day

Rather than wrestle with the error, I pivoted to an official Docker image of Playwright, and skipped installation of dependency:

mcr.microsoft.com/playwright/python:v1.39.0-jammy

docker image

Let’s dig down the issue:

The Compatibility Issue

Playwright demands compatibility. It officially supports Python versions 3.8 and higher, and it requires specific Linux distributions:

• Debian 11 (bullseye)
• Debian 12 (bookworm)
• Ubuntu 20.04 (focal)
• Ubuntu 22.04 (jammy)

However, on docker environment, the official image is only based on Unbuntu .

Use ENV PLAYWRIGHT_BROWSERS_PATH

After some search and experiments, I found the only solution in to specify the chromium binary files using ENV PLAYWRIGHT_BROWSERS_PATH during install . source code Dockerfile also use this environment variable to specify the broswer executable path.

using python:3.11-slim-bookworm

FROM python:3.11-slim-bookworm

ENV PLAYWRIGHT_BROWSERS_PATH=/app/ms-playwright
# Turns off buffering for easier container logging
ENV PYTHONUNBUFFERED=1

# Install git
RUN apt-get update

# install playwright or whatever
RUN python -m pip install --upgrade pip && \
    pip install -r requirements.txt

# install chrominum
RUN PLAYWRIGHT_BROWSERS_PATH=/app/ms-playwright && \
    playwright install --with-deps chromium

using ubuntu:22.04

FROM ubuntu:22.04

ENV PLAYWRIGHT_BROWSERS_PATH=/app/ms-playwright
# Turns off buffering for easier container logging
ENV PYTHONUNBUFFERED=1

# Install git
RUN apt-get update && \
    apt-get install -y python3-all python-is-python3 python3-pip

# install playwright or whatever
RUN python -m pip install --upgrade pip && \
    pip install -r requirements.txt

# install chrominum
RUN PLAYWRIGHT_BROWSERS_PATH=/app/ms-playwright && \
    playwright install --with-deps chromium

Memory requirement of Google Cloud Run

The playwright 1.39.0 requires slightly more than 512MB of memory to run on Google Cloud Run. Adjust the memory limit on GCR, as it’s 512 MB by default.

Conclusion

Use the official Docker image to save time, or specify the PLAYWRIGHT_BROWSERS_PATH environment variable on a supported linux docker image.

Further reading:

• A deep dive into the “official” Docker image for Python .
• Managing browser binaries