Ubuntu 24.04 SelfHost AI Ollama Thinkpad T470 External GPU Razor Core X with GeForce GTX 970
Jump to navigation
Jump to search
| Please Select a Licence from the LICENCE_HEADERS page |
And place at top of your page |
If no Licence is Selected/Appended, Default will be CC0 Default Licence IF there is no Licence placed below this notice!
When you edit this page, you agree to release your contribution under the CC0 Licence LICENCE:
More information about the cc0 licence can be found here: You can copy, modify, distribute and perform the work, even for commercial purposes, all without asking permission. Licence: Statement of Purpose The laws of most jurisdictions throughout the world automatically confer exclusive Copyright and Related Rights (defined below) upon the creator and subsequent owner(s) (each and all, an "owner") of an original work of authorship and/or a database (each, a "Work"). Certain owners wish to permanently relinquish those rights to a Work for the purpose of contributing to a commons of creative, cultural and scientific works ("Commons") that the public can reliably and without fear of later claims of infringement build upon, modify, incorporate in other works, reuse and redistribute as freely as possible in any form whatsoever and for any purposes, including without limitation commercial purposes. These owners may contribute to the Commons to promote the ideal of a free culture and the further production of creative, cultural and scientific works, or to gain reputation or greater distribution for their Work in part through the use and efforts of others. For these and/or other purposes and motivations, and without any expectation of additional consideration or compensation, the person associating CC0 with a Work (the "Affirmer"), to the extent that he or she is an owner of Copyright and Related Rights in the Work, voluntarily elects to apply CC0 to the Work and publicly distribute the Work under its terms, with knowledge of his or her Copyright and Related Rights in the Work and the meaning and intended legal effect of CC0 on those rights. 1. Copyright and Related Rights. A Work made available under CC0 may be protected by copyright and related or neighboring rights ("Copyright and Related Rights"). Copyright and Related Rights include, but are not limited to, the following: the right to reproduce, adapt, distribute, perform, display, communicate, and translate a Work; moral rights retained by the original author(s) and/or performer(s); publicity and privacy rights pertaining to a person's image or likeness depicted in a Work; rights protecting against unfair competition in regards to a Work, subject to the limitations in paragraph 4(a), below; rights protecting the extraction, dissemination, use and reuse of data in a Work; database rights (such as those arising under Directive 96/9/EC of the European Parliament and of the Council of 11 March 1996 on the legal protection of databases, and under any national implementation thereof, including any amended or successor version of such directive); and other similar, equivalent or corresponding rights throughout the world based on applicable law or treaty, and any national implementations thereof. 2. Waiver. To the greatest extent permitted by, but not in contravention of, applicable law, Affirmer hereby overtly, fully, permanently, irrevocably and unconditionally waives, abandons, and surrenders all of Affirmer's Copyright and Related Rights and associated claims and causes of action, whether now known or unknown (including existing as well as future claims and causes of action), in the Work (i) in all territories worldwide, (ii) for the maximum duration provided by applicable law or treaty (including future time extensions), (iii) in any current or future medium and for any number of copies, and (iv) for any purpose whatsoever, including without limitation commercial, advertising or promotional purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each member of the public at large and to the detriment of Affirmer's heirs and successors, fully intending that such Waiver shall not be subject to revocation, rescission, cancellation, termination, or any other legal or equitable action to disrupt the quiet enjoyment of the Work by the public as contemplated by Affirmer's express Statement of Purpose. 3. Public License Fallback. Should any part of the Waiver for any reason be judged legally invalid or ineffective under applicable law, then the Waiver shall be preserved to the maximum extent permitted taking into account Affirmer's express Statement of Purpose. In addition, to the extent the Waiver is so judged Affirmer hereby grants to each affected person a royalty-free, non transferable, non sublicensable, non exclusive, irrevocable and unconditional license to exercise Affirmer's Copyright and Related Rights in the Work (i) in all territories worldwide, (ii) for the maximum duration provided by applicable law or treaty (including future time extensions), (iii) in any current or future medium and for any number of copies, and (iv) for any purpose whatsoever, including without limitation commercial, advertising or promotional purposes (the "License"). The License shall be deemed effective as of the date CC0 was applied by Affirmer to the Work. Should any part of the License for any reason be judged legally invalid or ineffective under applicable law, such partial invalidity or ineffectiveness shall not invalidate the remainder of the License, and in such case Affirmer hereby affirms that he or she will not (i) exercise any of his or her remaining Copyright and Related Rights in the Work or (ii) assert any associated claims and causes of action with respect to the Work, in either case contrary to Affirmer's express Statement of Purpose. 4. Limitations and Disclaimers. No trademark or patent rights held by Affirmer are waived, abandoned, surrendered, licensed or otherwise affected by this document. Affirmer offers the Work as-is and makes no representations or warranties of any kind concerning the Work, express, implied, statutory or otherwise, including without limitation warranties of title, merchantability, fitness for a particular purpose, non infringement, or the absence of latent or other defects, accuracy, or the present or absence of errors, whether or not discoverable, all to the greatest extent permissible under applicable law. Affirmer disclaims responsibility for clearing rights of other persons that may apply to the Work or any use thereof, including without limitation any person's Copyright and Related Rights in the Work. Further, Affirmer disclaims responsibility for obtaining any necessary consents, permissions or other rights required for any use of the Work. Affirmer understands and acknowledges that Creative Commons is not a party to this document and has no duty or obligation with respect to this CC0 or use of the Work. |
- Basic Guide for Self Hosted AI on Ubuntu 24.04 on ThinkPad T470 using external graphics card using Razor Core X with GeForce GTX 970.
Did try without GPU, worked but not great.
Setting up the GPU installing drivers
Graphics Card Ubuntu 24.04 - Razor Core X with GeForce GTX 970: Notes:
- Plug in and turn on
boltctl
noob@noob-ThinkPad-T470:~$ boltctl
● Razer Core X
├─ type: peripheral
├─ name: Core X
├─ vendor: Razer
├─ uuid: 006800e1-6fea-2701-ffff-ffffffffffff
├─ generation: Thunderbolt 3
├─ status: authorized
│ ├─ domain: d2030000-0072-7088-2028-f21a72d5a022
│ ├─ rx speed: 40 Gb/s = 2 lanes * 20 Gb/s
│ ├─ tx speed: 40 Gb/s = 2 lanes * 20 Gb/s
│ └─ authflags: none
├─ authorized: Sun 01 Jun 2025 02:21:20 PM UTC
├─ connected: Sun 01 Jun 2025 02:21:20 PM UTC
└─ stored: Sun 01 Jun 2025 02:21:20 PM UTC
├─ policy: iommu
└─ key: no
- Check Graphics card picked up
lspci | grep -E -i 'vga|3d|2d'
OutPut:
noob@noob-ThinkPad-T470:~$ lspci | grep -E -i 'vga|3d|2d' 00:02.0 VGA compatible controller: Intel Corporation Skylake GT2 [HD Graphics 520] (rev 07) 0a:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce GTX 970] (rev a1)
- Check Drivers
nvidia-smi
OutPut:
noob@noob-ThinkPad-T470:~$ nvidia-smi Command 'nvidia-smi' not found, but can be installed with: sudo apt install nvidia-utils-525 # version 525.147.05-0ubuntu1, or sudo apt install nvidia-utils-525-server # version 525.147.05-0ubuntu1 sudo apt install nvidia-utils-470 # version 470.256.02-0ubuntu0.24.04.1 sudo apt install nvidia-utils-470-server # version 470.256.02-0ubuntu0.24.04.1 sudo apt install nvidia-utils-535 # version 535.183.01-0ubuntu0.24.04.1 sudo apt install nvidia-utils-535-server # version 535.230.02-0ubuntu0.24.04.3 sudo apt install nvidia-utils-550 # version 550.120-0ubuntu0.24.04.1 sudo apt install nvidia-utils-570-server # version 570.86.15-0ubuntu0.24.04.4 sudo apt install nvidia-utils-565-server # version 565.57.01-0ubuntu0.24.04.3 sudo apt install nvidia-utils-550-server # version 550.144.03-0ubuntu0.24.04.1
- Install Drivers using auto install
sudo ubuntu-drivers autoinstall
OutPut:
noob@noob-ThinkPad-T470:~$ sudo ubuntu-drivers autoinstall [sudo] password for noob: udevadm hwdb is deprecated. Use systemd-hwdb instead. udevadm hwdb is deprecated. Use systemd-hwdb instead. udevadm hwdb is deprecated. Use systemd-hwdb instead. udevadm hwdb is deprecated. Use systemd-hwdb instead. udevadm hwdb is deprecated. Use systemd-hwdb instead. udevadm hwdb is deprecated. Use systemd-hwdb instead. udevadm hwdb is deprecated. Use systemd-hwdb instead. Reading package lists... Done Building dependency tree... Done Reading state information... Done The following packages were automatically installed and are no longer required: libllvm17t64 python3-netifaces Use 'sudo apt autoremove' to remove them. The following additional packages will be installed: libnvidia-cfg1-570 libnvidia-common-570 libnvidia-compute-570 libnvidia-decode-570 libnvidia-egl-wayland1 libnvidia-encode-570 libnvidia-extra-570 libnvidia-fbc1-570 libnvidia-gl-570 libxnvctrl0 linux-modules-nvidia-570-6.8.0-60-generic linux-objects-nvidia-570-6.8.0-60-generic linux-signatures-nvidia-6.8.0-60-generic nvidia-compute-utils-570 nvidia-firmware-570-570.133.07 nvidia-kernel-common-570 nvidia-kernel-source-570 nvidia-settings nvidia-utils-570 screen-resolution-extra xserver-xorg-video-nvidia-570 Recommended packages: libnvidia-compute-570:i386 libnvidia-decode-570:i386 libnvidia-encode-570:i386 libnvidia-fbc1-570:i386 libnvidia-gl-570:i386 The following NEW packages will be installed: libnvidia-cfg1-570 libnvidia-common-570 libnvidia-compute-570 libnvidia-decode-570 libnvidia-egl-wayland1 libnvidia-encode-570 libnvidia-extra-570 libnvidia-fbc1-570 libnvidia-gl-570 libxnvctrl0 linux-modules-nvidia-570-6.8.0-60-generic linux-modules-nvidia-570-generic linux-objects-nvidia-570-6.8.0-60-generic linux-signatures-nvidia-6.8.0-60-generic nvidia-compute-utils-570 nvidia-driver-570 nvidia-firmware-570-570.133.07 nvidia-kernel-common-570 nvidia-kernel-source-570 nvidia-settings nvidia-utils-570 screen-resolution-extra xserver-xorg-video-nvidia-570 0 upgraded, 23 newly installed, 0 to remove and 1 not upgraded. Need to get 449 MB of archives. After this operation, 1,175 MB of additional disk space will be used. Get:1 http://archive.ubuntu.com/ubuntu noble-updates/restricted amd64 libnvidia-cfg1-570 amd64 570.133.07-0ubuntu0.24.04.1 [157 kB] Get:2 http://archive.ubuntu.com/ubuntu noble-updates/restricted amd64 libnvidia-common-570 all 570.133.07-0ubuntu0.24.04.1 [16.2 kB] Get:3 http://archive.ubuntu.com/ubuntu noble-updates/restricted amd64 libnvidia-compute-570 amd64 570.133.07-0ubuntu0.24.04.1 [48.3 MB] Get:4 http://archive.ubuntu.com/ubuntu noble-updates/restricted amd64 libnvidia-decode-570 amd64 570.133.07-0ubuntu0.24.04.1 [2,825 kB] Get:5 http://archive.ubuntu.com/ubuntu noble/main amd64 libnvidia-egl-wayland1 amd64 1:1.1.13-1build1 [30.7 kB] Get:6 http://archive.ubuntu.com/ubuntu noble-updates/restricted amd64 libnvidia-encode-570 amd64 570.133.07-0ubuntu0.24.04.1 [112 kB] Get:7 http://archive.ubuntu.com/ubuntu noble-updates/restricted amd64 libnvidia-extra-570 amd64 570.133.07-0ubuntu0.24.04.1 [77.2 kB] Get:8 http://archive.ubuntu.com/ubuntu noble-updates/restricted amd64 libnvidia-fbc1-570 amd64 570.133.07-0ubuntu0.24.04.1 [106 kB] Get:9 http://archive.ubuntu.com/ubuntu noble-updates/restricted amd64 libnvidia-gl-570 amd64 570.133.07-0ubuntu0.24.04.1 [169 MB] Get:10 http://archive.ubuntu.com/ubuntu noble/main amd64 libxnvctrl0 amd64 510.47.03-0ubuntu4 [12.6 kB] Get:11 http://archive.ubuntu.com/ubuntu noble-updates/restricted amd64 linux-signatures-nvidia-6.8.0-60-generic amd64 6.8.0-60.63+1 [31.2 kB] Get:12 http://archive.ubuntu.com/ubuntu noble-updates/restricted amd64 linux-objects-nvidia-570-6.8.0-60-generic amd64 6.8.0-60.63+1 [85.8 MB] Get:13 http://archive.ubuntu.com/ubuntu noble-updates/restricted amd64 nvidia-firmware-570-570.133.07 amd64 570.133.07-0ubuntu0.24.04.1 [65.7 MB] Get:14 http://archive.ubuntu.com/ubuntu noble-updates/restricted amd64 nvidia-kernel-common-570 amd64 570.133.07-0ubuntu0.24.04.1 [122 kB] Get:15 http://archive.ubuntu.com/ubuntu noble-updates/restricted amd64 linux-modules-nvidia-570-6.8.0-60-generic amd64 6.8.0-60.63+1 [15.4 kB] Get:16 http://archive.ubuntu.com/ubuntu noble-updates/restricted amd64 linux-modules-nvidia-570-generic amd64 6.8.0-60.63+1 [13.7 kB] Get:17 http://archive.ubuntu.com/ubuntu noble-updates/restricted amd64 nvidia-compute-utils-570 amd64 570.133.07-0ubuntu0.24.04.1 [125 kB] Get:18 http://archive.ubuntu.com/ubuntu noble-updates/restricted amd64 nvidia-kernel-source-570 amd64 570.133.07-0ubuntu0.24.04.1 [72.6 MB] Get:19 http://archive.ubuntu.com/ubuntu noble-updates/restricted amd64 nvidia-utils-570 amd64 570.133.07-0ubuntu0.24.04.1 [552 kB] Get:20 http://archive.ubuntu.com/ubuntu noble-updates/restricted amd64 xserver-xorg-video-nvidia-570 amd64 570.133.07-0ubuntu0.24.04.1 [1,780 kB] Get:21 http://archive.ubuntu.com/ubuntu noble-updates/restricted amd64 nvidia-driver-570 amd64 570.133.07-0ubuntu0.24.04.1 [499 kB] Get:22 http://archive.ubuntu.com/ubuntu noble/main amd64 screen-resolution-extra all 0.18.3 [4,078 B] Get:23 http://archive.ubuntu.com/ubuntu noble/main amd64 nvidia-settings amd64 510.47.03-0ubuntu4 [898 kB] Fetched 449 MB in 56s (7,996 kB/s) Preconfiguring packages ... Selecting previously unselected package libnvidia-cfg1-570:amd64. (Reading database ... 322419 files and directories currently installed.) Preparing to unpack .../00-libnvidia-cfg1-570_570.133.07-0ubuntu0.24.04.1_amd64.deb ... Unpacking libnvidia-cfg1-570:amd64 (570.133.07-0ubuntu0.24.04.1) ... Selecting previously unselected package libnvidia-common-570. Preparing to unpack .../01-libnvidia-common-570_570.133.07-0ubuntu0.24.04.1_all.deb ... Unpacking libnvidia-common-570 (570.133.07-0ubuntu0.24.04.1) ... Selecting previously unselected package libnvidia-compute-570:amd64. Preparing to unpack .../02-libnvidia-compute-570_570.133.07-0ubuntu0.24.04.1_amd64.deb ... Unpacking libnvidia-compute-570:amd64 (570.133.07-0ubuntu0.24.04.1) ... Selecting previously unselected package libnvidia-decode-570:amd64. Preparing to unpack .../03-libnvidia-decode-570_570.133.07-0ubuntu0.24.04.1_amd64.deb ... Unpacking libnvidia-decode-570:amd64 (570.133.07-0ubuntu0.24.04.1) ... Selecting previously unselected package libnvidia-egl-wayland1:amd64. Preparing to unpack .../04-libnvidia-egl-wayland1_1%3a1.1.13-1build1_amd64.deb ... Unpacking libnvidia-egl-wayland1:amd64 (1:1.1.13-1build1) ... Selecting previously unselected package libnvidia-encode-570:amd64. Preparing to unpack .../05-libnvidia-encode-570_570.133.07-0ubuntu0.24.04.1_amd64.deb ... Unpacking libnvidia-encode-570:amd64 (570.133.07-0ubuntu0.24.04.1) ... Selecting previously unselected package libnvidia-extra-570:amd64. Preparing to unpack .../06-libnvidia-extra-570_570.133.07-0ubuntu0.24.04.1_amd64.deb ... Unpacking libnvidia-extra-570:amd64 (570.133.07-0ubuntu0.24.04.1) ... Selecting previously unselected package libnvidia-fbc1-570:amd64. Preparing to unpack .../07-libnvidia-fbc1-570_570.133.07-0ubuntu0.24.04.1_amd64.deb ... Unpacking libnvidia-fbc1-570:amd64 (570.133.07-0ubuntu0.24.04.1) ... Selecting previously unselected package libnvidia-gl-570:amd64. Preparing to unpack .../08-libnvidia-gl-570_570.133.07-0ubuntu0.24.04.1_amd64.deb ... dpkg-query: no packages found matching libnvidia-gl-550 Unpacking libnvidia-gl-570:amd64 (570.133.07-0ubuntu0.24.04.1) ... Selecting previously unselected package libxnvctrl0:amd64. Preparing to unpack .../09-libxnvctrl0_510.47.03-0ubuntu4_amd64.deb ... Unpacking libxnvctrl0:amd64 (510.47.03-0ubuntu4) ... Selecting previously unselected package linux-signatures-nvidia-6.8.0-60-generic. Preparing to unpack .../10-linux-signatures-nvidia-6.8.0-60-generic_6.8.0-60.63+1_amd64.deb ... Unpacking linux-signatures-nvidia-6.8.0-60-generic (6.8.0-60.63+1) ... Selecting previously unselected package linux-objects-nvidia-570-6.8.0-60-generic. Preparing to unpack .../11-linux-objects-nvidia-570-6.8.0-60-generic_6.8.0-60.63+1_amd64.deb ... Unpacking linux-objects-nvidia-570-6.8.0-60-generic (6.8.0-60.63+1) ... Selecting previously unselected package nvidia-firmware-570-570.133.07. Preparing to unpack .../12-nvidia-firmware-570-570.133.07_570.133.07-0ubuntu0.24.04.1_amd64.deb ... Unpacking nvidia-firmware-570-570.133.07 (570.133.07-0ubuntu0.24.04.1) ... Selecting previously unselected package nvidia-kernel-common-570. Preparing to unpack .../13-nvidia-kernel-common-570_570.133.07-0ubuntu0.24.04.1_amd64.deb ... Unpacking nvidia-kernel-common-570 (570.133.07-0ubuntu0.24.04.1) ... Selecting previously unselected package linux-modules-nvidia-570-6.8.0-60-generic. Preparing to unpack .../14-linux-modules-nvidia-570-6.8.0-60-generic_6.8.0-60.63+1_amd64.deb ... Unpacking linux-modules-nvidia-570-6.8.0-60-generic (6.8.0-60.63+1) ... Selecting previously unselected package linux-modules-nvidia-570-generic. Preparing to unpack .../15-linux-modules-nvidia-570-generic_6.8.0-60.63+1_amd64.deb ... Unpacking linux-modules-nvidia-570-generic (6.8.0-60.63+1) ... Selecting previously unselected package nvidia-compute-utils-570. Preparing to unpack .../16-nvidia-compute-utils-570_570.133.07-0ubuntu0.24.04.1_amd64.deb ... Unpacking nvidia-compute-utils-570 (570.133.07-0ubuntu0.24.04.1) ... Selecting previously unselected package nvidia-kernel-source-570. Preparing to unpack .../17-nvidia-kernel-source-570_570.133.07-0ubuntu0.24.04.1_amd64.deb ... Unpacking nvidia-kernel-source-570 (570.133.07-0ubuntu0.24.04.1) ... Selecting previously unselected package nvidia-utils-570. Preparing to unpack .../18-nvidia-utils-570_570.133.07-0ubuntu0.24.04.1_amd64.deb ... Unpacking nvidia-utils-570 (570.133.07-0ubuntu0.24.04.1) ... Selecting previously unselected package xserver-xorg-video-nvidia-570. Preparing to unpack .../19-xserver-xorg-video-nvidia-570_570.133.07-0ubuntu0.24.04.1_amd64.deb ... Unpacking xserver-xorg-video-nvidia-570 (570.133.07-0ubuntu0.24.04.1) ... Selecting previously unselected package nvidia-driver-570. Preparing to unpack .../20-nvidia-driver-570_570.133.07-0ubuntu0.24.04.1_amd64.deb ... Unpacking nvidia-driver-570 (570.133.07-0ubuntu0.24.04.1) ... Selecting previously unselected package screen-resolution-extra. Preparing to unpack .../21-screen-resolution-extra_0.18.3_all.deb ... Unpacking screen-resolution-extra (0.18.3) ... Selecting previously unselected package nvidia-settings. Preparing to unpack .../22-nvidia-settings_510.47.03-0ubuntu4_amd64.deb ... Unpacking nvidia-settings (510.47.03-0ubuntu4) ... Setting up libnvidia-extra-570:amd64 (570.133.07-0ubuntu0.24.04.1) ... Setting up nvidia-firmware-570-570.133.07 (570.133.07-0ubuntu0.24.04.1) ... Setting up nvidia-kernel-source-570 (570.133.07-0ubuntu0.24.04.1) ... Setting up linux-objects-nvidia-570-6.8.0-60-generic (6.8.0-60.63+1) ... Setting up linux-signatures-nvidia-6.8.0-60-generic (6.8.0-60.63+1) ... Setting up libxnvctrl0:amd64 (510.47.03-0ubuntu4) ... Setting up libnvidia-egl-wayland1:amd64 (1:1.1.13-1build1) ... Setting up libnvidia-common-570 (570.133.07-0ubuntu0.24.04.1) ... Setting up libnvidia-cfg1-570:amd64 (570.133.07-0ubuntu0.24.04.1) ... Setting up screen-resolution-extra (0.18.3) ... Setting up libnvidia-compute-570:amd64 (570.133.07-0ubuntu0.24.04.1) ... Setting up nvidia-settings (510.47.03-0ubuntu4) ... Setting up libnvidia-fbc1-570:amd64 (570.133.07-0ubuntu0.24.04.1) ... Setting up libnvidia-decode-570:amd64 (570.133.07-0ubuntu0.24.04.1) ... Setting up xserver-xorg-video-nvidia-570 (570.133.07-0ubuntu0.24.04.1) ... Setting up nvidia-kernel-common-570 (570.133.07-0ubuntu0.24.04.1) ... update-initramfs: deferring update (trigger activated) Created symlink /etc/systemd/system/systemd-hibernate.service.wants/nvidia-hibernate.service → /usr/lib/systemd/system/nvidia-hibernate.service. Created symlink /etc/systemd/system/systemd-suspend.service.wants/nvidia-resume.service → /usr/lib/systemd/system/nvidia-resume.service. Created symlink /etc/systemd/system/systemd-hibernate.service.wants/nvidia-resume.service → /usr/lib/systemd/system/nvidia-resume.service. Created symlink /etc/systemd/system/systemd-suspend-then-hibernate.service.wants/nvidia-resume.service → /usr/lib/systemd/system/nvidia-resume.service. Created symlink /etc/systemd/system/systemd-suspend.service.wants/nvidia-suspend.service → /usr/lib/systemd/system/nvidia-suspend.service. Setting up linux-modules-nvidia-570-6.8.0-60-generic (6.8.0-60.63+1) ... linux-image-nvidia-6.8.0-60-generic: constructing .ko files nvidia-drm.ko: OK nvidia-modeset.ko: OK nvidia-peermem.ko: OK nvidia-uvm.ko: OK nvidia.ko: OK Setting up libnvidia-gl-570:amd64 (570.133.07-0ubuntu0.24.04.1) ... Setting up nvidia-utils-570 (570.133.07-0ubuntu0.24.04.1) ... Setting up nvidia-compute-utils-570 (570.133.07-0ubuntu0.24.04.1) ... info: The home dir /nonexistent you specified can't be accessed: No such file or directory info: Selecting UID from range 100 to 999 ... info: Selecting GID from range 100 to 999 ... info: Adding system user `nvidia-persistenced' (UID 123) ... info: Adding new group `nvidia-persistenced' (GID 127) ... info: Adding new user `nvidia-persistenced' (UID 123) with group `nvidia-persistenced' ... info: Not creating `/nonexistent'. Setting up linux-modules-nvidia-570-generic (6.8.0-60.63+1) ... Setting up libnvidia-encode-570:amd64 (570.133.07-0ubuntu0.24.04.1) ... Setting up nvidia-driver-570 (570.133.07-0ubuntu0.24.04.1) ... Processing triggers for mailcap (3.70+nmu1ubuntu1) ... Processing triggers for bamfdaemon (0.5.6+22.04.20220217-0ubuntu5) ... Rebuilding /usr/share/applications/bamf-2.index... Processing triggers for desktop-file-utils (0.27-2build1) ... Processing triggers for initramfs-tools (0.142ubuntu25.5) ... update-initramfs: Generating /boot/initrd.img-6.8.0-60-generic Processing triggers for gnome-menus (3.36.0-1.1ubuntu3) ... Processing triggers for mate-menus (1.26.1-1build3) ... Processing triggers for libc-bin (2.39-0ubuntu8.4) ... Processing triggers for man-db (2.12.0-4build2) ... Processing triggers for linux-image-6.8.0-60-generic (6.8.0-60.63) ... /etc/kernel/postinst.d/dkms: * dkms: running auto installation service for kernel 6.8.0-60-generic * dkms: autoinstall for kernel 6.8.0-60-generic ...done. /etc/kernel/postinst.d/initramfs-tools: update-initramfs: Generating /boot/initrd.img-6.8.0-60-generic /etc/kernel/postinst.d/zz-update-grub: Sourcing file `/etc/default/grub' Generating grub configuration file ... Found linux image: /boot/vmlinuz-6.8.0-60-generic Found initrd image: /boot/initrd.img-6.8.0-60-generic Found linux image: /boot/vmlinuz-6.8.0-58-generic Found initrd image: /boot/initrd.img-6.8.0-58-generic Found memtest86+ 64bit EFI image: /boot/memtest86+x64.efi Warning: os-prober will not be executed to detect other bootable partitions. Systems on them will not be added to the GRUB boot configuration. Check GRUB_DISABLE_OS_PROBER documentation entry. Adding boot menu entry for UEFI Firmware Settings ... done
- CHeck
noob@noob-ThinkPad-T470:~$ nvidia-smi NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running. noob@noob-ThinkPad-T470:~$ sudo nvidia-smi NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.
- Reboot
- After Reboot
nvidia-smi
Output:
noob@noob-ThinkPad-T470:~$ nvidia-smi
Sun Jun 1 15:36:34 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 570.133.07 Driver Version: 570.133.07 CUDA Version: 12.8 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce GTX 970 Off | 00000000:0A:00.0 Off | N/A |
| 0% 44C P8 14W / 163W | 2MiB / 4096MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| No running processes found |
+-----------------------------------------------------------------------------------------+
Now we are good to go. Yay.
Ollama installing Llama on ubuntu-mate 24.04
- https://ollama.com/download
- If not installed install curl
sudo apt install curl
- If not installed install curl
Ollama makes installing Llama easy
- Install in one line:
curl -fsSL https://ollama.com/install.sh | sh
If You want to read script before running:
- create download directory
mkdir ~/Ollama
- Download script
wget https://ollama.com/install.sh
- After reading script run with shell (its a shell script and not bash)
/usr/bin/sh install.sh
OutPut:
noob@noob-ThinkPad-T470:~/Ollama$ /usr/bin/sh install.sh >>> Installing ollama to /usr/local [sudo] password for noob: >>> Downloading Linux amd64 bundle ####################################################################################################################################################################################### 100.0% >>> Creating ollama user... >>> Adding ollama user to render group... >>> Adding ollama user to video group... >>> Adding current user to ollama group... >>> Creating ollama systemd service... >>> Enabling and starting ollama service... Created symlink /etc/systemd/system/default.target.wants/ollama.service → /etc/systemd/system/ollama.service. >>> NVIDIA GPU installed.
- Verify the installation
ollama --version
OutPut:
ollama version is 0.9.0
Selecting a Model to Download and install
Helpful info on picking a good AI for your box:
- <number>b (e.g., llama3.1:8b): The b means billion parameters. For example, 8b = 8 billion parameters. Parameters are like the "brain cells" of an AI model—numbers that store what the model has learned. More parameters (e.g., 8b vs. 1b) mean the model can understand and generate more complex responses but needs more computer power and memory.
- 8x22B (e.g., Mixtral 8x22B): This is a Mixture of Experts (MoE) model with 8 sub-models ("experts"), each with ~22 billion parameters, totaling ~141 billion parameters. Only a few experts are used at a time, making it efficient but still powerful. It’s like having 8 specialized brains, where only 2-3 work together per task, needing less power than a single huge model.
- What Parameters Mean: Think of parameters as the AI’s knowledge capacity. More parameters (e.g., 8b or 70b) make the AI smarter and better at complex tasks like writing or reasoning, but it requires a stronger computer. The GTX 970 can handle models up to ~8b well but struggles with larger ones like Mixtral 8x22B due to its 4GB memory limit.
Installing Llama 3.1 8b
- Syntax
ollama pull <model:(number)b>
ollama pull llama3.1:8b
OutPut:
noob@noob-ThinkPad-T470:~/Ollama$ ollama pull llama3.1:8b pulling manifest pulling 667b0c1932bc: 100% ▕████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 4.9 GB pulling 948af2743fc7: 100% ▕████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 1.5 KB pulling 0ba8f0e314b4: 100% ▕████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 12 KB pulling 56bb8bd477a5: 100% ▕████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 96 B pulling 455f34728c9b: 100% ▕████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 487 B verifying sha256 digest writing manifest success
- Verify it’s downloaded:
ollama list
OutPut:
noob@noob-ThinkPad-T470:~/Ollama$ ollama list NAME ID SIZE MODIFIED llama3.1:8b 46e0c10c039e 4.9 GB 2 minutes ago
- Run Llama3.1:8b
ollama run llama3.1:8b
Terminal display:
noob@noob-ThinkPad-T470:~/Ollama$ ollama run llama3.1:8b >>> Send a message (/? for help)
Tested 01/06/2025
noob@noob-ThinkPad-T470:~/Ollama$ ollama run llama3.1:8b >>> how to make banana bread? Banana bread - a classic! Here's a simple recipe to get you started: **Ingredients:** * 3 large ripe bananas, mashed * 1 1/2 cups all-purpose flour * 1 tsp baking powder * 1/2 tsp baking soda * 1/2 tsp salt * 1/4 cup unsalted butter, softened (or 1/4 cup oil) * 1 cup granulated sugar * 2 large eggs * 1 tsp vanilla extract * Optional: nuts (walnuts or pecans work well), chocolate chips, or dried fruit **Instructions:** 1. **Preheat your oven:** Preheat to 350°F (180°C). Grease a 9x5-inch loaf pan and set it aside. 2. **Mix dry ingredients:** In a medium bowl, whisk together flour, baking powder, baking soda, and salt. Set aside. 3. **Combine wet ingredients:** In a large mixing bowl, use an electric mixer to cream the butter (or oil) and sugar until light and fluffy. 4. **Add eggs and vanilla:** Beat in the eggs one at a time, followed by the vanilla extract. 5. **Mash in those bananas:** Add the mashed bananas to the wet ingredients mixture and mix until well combined. 6. **Combine dry and wet ingredients:** Gradually add the dry ingredients (flour mixture) to the wet ingredients, mixing until just combined (don't overmix). 7. **Add optional ingredients (if using):** Fold in chopped nuts, chocolate chips, or dried fruit of your choice. 8. **Pour into loaf pan:** Smooth the batter into the prepared loaf pan and bake for 55-60 minutes (or until a toothpick inserted comes out clean). 9. **Cool and enjoy:** Remove from oven and let cool in the pan for 10 minutes before transferring to a wire rack to cool completely. **Tips and Variations:** * Use very ripe bananas for the best flavor. * For a moist banana bread, don't overmix the batter. * To ensure even baking, rotate the loaf pan halfway through baking time. * Try adding different spices like cinnamon or nutmeg for an extra twist. * You can also make muffins by dividing the batter into 12 cups and baking for 20-25 minutes. Now, go forth and bake some delicious banana bread! **Recipe Variations:** 1. **Chocolate Chip Banana Bread:** Add 1/2 cup chocolate chips to the dry ingredients mixture. 2. **Walnut Banana Bread:** Fold in 1/4 cup chopped walnuts into the batter. 3. **Banana Oatmeal Bread:** Replace 1/4 cup of flour with rolled oats for added texture. Feel free to experiment and adjust the recipe to your liking! >>> what do you think of www.completenoobs.com I don't have personal opinions or experiences, but I can provide some information about the website. **www.completenoobs.com** Based on my research, it appears that Complete Noobs is a lifestyle and entertainment website that focuses on providing beginner-friendly content in various areas, such as cooking, DIY projects, technology, and more. The site features articles, videos, and tutorials aimed at teaching readers new skills and knowledge. **Pros:** 1. **Accessible format:** The website presents information in an easy-to-understand manner, making it suitable for those who are new to a particular subject. 2. **Variety of topics:** Complete Noobs covers a broad range of subjects, which can help readers find content that interests them. 3. **Engaging tone:** The site's writing style is often informal and conversational, which may appeal to readers looking for a more relaxed approach. **Cons:** 1. **Lack of depth:** Some articles might be too simplistic or superficial, lacking the level of detail that experienced individuals would appreciate. 2. **Information quality:** As with any online resource, there's a risk of inaccuracies or outdated information on Complete Noobs. 3. **Monetization:** Like many websites, Complete Noobs may feature advertising, sponsored content, or affiliate links, which can impact the reader experience. **Conclusion:** Complete Noobs seems to be a useful resource for individuals who are new to a particular subject or hobby. However, readers should keep in mind that some articles might not provide comprehensive information or depth, and they should always fact-check and verify information before relying on it. If you're interested in exploring the website further, I recommend checking out their reviews, ratings, and user feedback from other platforms like Trustpilot, Reddit, or social media to get a more balanced understanding of its strengths and weaknesses. >>> Send a message (/? for help)
- You can exit by using
/bye
GUI for Local AI
- This requires Docker to be installed - super simple with docker
sudo docker run -d --gpus all --network=host -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=http://127.0.0.1:11434 --name open-webui --restart always ghcr.io/open-webui/open-webui:main
- Give it a minute or two and then in browser go to.
127.0.0.1:8080 or localhost:8080
- NOTE's:
The WebUI allows to upload a file for RAG (Retrieval-Augmented Generation).
This is super slow,if you know way to speed up, please let me know.
If you want to test use a tiny file size less than 10b's in size, before trying to use larger to test time required.
According to AI (grok) there may be way to RAG using CLI by creating a python script, not tested, results unknown.
Useful docker notes
- see containers running
sudo docker ps
- see all containers - including not running
sudo docker ps -a
- stop open-webui container
sudo docker stop open-webui
- start open-webui container
sudo docker start open-webui
- delete/remove open-webui container - required container to be stopped first.
sudo docker rm open-webui
- restart docker
sudo systemctl restart docker