"1. Model Pruning"

"Reduce the size of the model by eliminating parameters."

"2. Quantization"

"Reduce the precision of the numerical values used within the model. For example, you can switch from float32 to float16 (or even further down to int8)."

"3. Model Distillation"

"Train a smaller model to imitate the behavior of a larger model."

"4. Parallel Processing"

"5. Subword Tokenization"

"Break words into smaller units (i.e. subwords). This will allow you to reduce the size of the vocabulary."

"6. Optimized Libraries"

"Use highly optimized libraries (like Nvidia's TensorRT) to run your AI workloads. It can significantly boost the performance."

"7. Batch Inference Workloads"

"A good chunk of the chip's memory bandwidth is consumed by the model parameters that you load."

"You don't have to load model parameters for every input sequence. You can batch them together and load the parameters only once."

"8. Adapters"

"They are compact additional layers in the model (e.g. LoRa, QLoRa). These layers are tunable, which means you can train them to do what you want. You can make these layers lightweight, which helps the model to learn quickly."

"These are AI-infused products that are designed to solve specific problems in a particular vertical."

"Verticalized AI models are specific to that domain and cannot really do much outside of that domain."

"1. Luxury of dealing only with domain-specific data"

"2. Infusing domain knowledge into the product"

"3. Domain expertise of the people"

"4. Integrating with existing systems"

"5. Customer willingness to pay is high"

"6. Value capture is high"

"Because it will cost way more for the customer to go out and use disjointed tools."

"7. Intra-domain network effects"

"8. Snowball effect of product usage data"

"Verticalized AI is particularly well suited to the enterprise. And companies that build verticalized AI applications are poised to win big time."