• My Installation of WSL Linux & Docker

    Home » Forums » AskWoody support » Windows » Windows 11 » Windows 11 version 23H2 » My Installation of WSL Linux & Docker

    Author
    Topic
    #2707672

    I’m running Windows 11 Pro Version 23H2 (OS Build 22631.4169).  My goal is to have local offline AI running on Ubuntu 24.04.  Here’s how I’ve started:

    The first order of business is in Control Panel > Turn Windows features on or off, you need to have Virtual Machine Platform and Windows Subsystem for Linux turned on.  Then you need to open an elevated Powershell terminal. If you want to install Linux on a drive other than C:, use

    set-location X: (where X is your preferred drive). Then type

    wsl –install

    —Update WSL from https://learn.microsoft.com/en-us/windows/wsl/install-manual#step-4—download-the-linux-kernel-update-package. In your browser, download the .msi package and install it in Windows.  Then, back in Powershell

    —Set the WSL default to Version 2

    wsl –set-default-version 2

    —Still in Powershell, download Linux app package.

    curl.exe -L -o Linux.appx https://wslstorestorage.blob.core.windows.net/wslblob/Ubuntu2404-240425.AppxBundle

    —Install the package.  Still in Powershell

    Add-AppxPackage .\Linux.appx

    —In Windows, double-click the .appx file to install Linux.  A Linux terminal will come on screen.  You will be asked for a username, and a password (twice on a new installation). Note that when typing in a password, the curser does not move or indicate any input with *.

    —From this point on, you’re in Linux, not Powershell (the Powershell console is still open, unless you close it).  Update Linux after the installation.

    sudo apt update (you’ll be asked for your password)

    sudo apt full-upgrade -y

    —Once installed, the Linux App (I installed Ubuntu) can be moved from the C: drive. First, exit the Linux terminal, then in an elevated Powershell, shutdown WSL

    wsl –shutdown

    —With WSL shutdown, moving Linux will also move the vhdx virtual file, so it won’t be slowly growing on your C: drive and eating up your drive free space.

    —Go to Settings > Apps > Installed Apps and scroll down to the Linux App. On the right side is the options icon (three stacked dots). Click that, and one of the options is Move. Select Move, and a dropdown will show the available drives. The move may fail the first time; if so, just try it again in a minute or two.

    —Moving Linux also makes it much easier to cleanup mistakes by making drive images of the installation partition at each successful step in installing additional software. If something goes awry, just restore the known-good image (which I’ve already done a number of times).

    —Your installation of Linux will appear in the Apps menu.  That’s where you can launch it.  To install Docker in Linux (Ubuntu) run this script in the Linux terminal:

    # Add Docker’s official GPG key:
    sudo apt-get update
    sudo apt-get install ca-certificates curl
    sudo install -m 0755 -d /etc/apt/keyrings
    sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
    sudo chmod a+r /etc/apt/keyrings/docker.asc

    # Add the repository to Apt sources:
    echo \
    “deb [arch=$(dpkg –print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
    $(. /etc/os-release && echo “$VERSION_CODENAME”) stable” | \
    sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
    sudo apt-get update

    —Next,

    sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

    —Under WSL, the iptables in Linux need to be changed to “legacy”.

    sudo update-alternatives –config iptables

    —Select “1” and then Enter.  That will get Docker to run.  I have a partition image of my Ubuntu installation at this point of success.

    Always create a fresh drive image before making system changes/Windows updates; you may need to start over!
    We all have our own reasons for doing the things that we do with our systems; we don't need anyone's approval, and we don't all have to do the same things.
    We were all once "Average Users".

    • This topic was modified 7 months, 2 weeks ago by bbearren.
    • This topic was modified 7 months, 2 weeks ago by bbearren.
    • This topic was modified 7 months, 2 weeks ago by bbearren.
    • This topic was modified 7 months, 2 weeks ago by bbearren.
    • This topic was modified 7 months, 2 weeks ago by bbearren.
    • This topic was modified 7 months, 2 weeks ago by bbearren.
    3 users thanked author for this post.
    Viewing 4 reply threads
    Author
    Replies
    • #2707730

      I had intended to put the following in my OP, but forgot it.

      There are many, many complete instruction sets on the internet for installing Linux and Docker, none of which worked in its entirety on my system (hence all the drive image restores).  The OP is a compilation of those steps that worked for me from all those sources.

      After a number of false starts, I started tracking the successful steps in a Notepad file.  When I finally got Linux & Docker up and running, I formatted my Linux partition, then followed those steps in the OP, and recreated my working environment.  YMMV.

      Always create a fresh drive image before making system changes/Windows updates; you may need to start over!
      We all have our own reasons for doing the things that we do with our systems; we don't need anyone's approval, and we don't all have to do the same things.
      We were all once "Average Users".

      • #2707747

        Installing Ollama with Docker, CPU Only (I don’t have a GPU—click Computer Specs in my signature line for my daily driver configuration).  Start your WSL Linux console, then

        sudo service docker start (you’ll be asked for your password)

        docker run -d -v ollama:/root/.ollama -p 11434:11434 –name ollama ollama/ollama

        —After Ollama has downloaded and installed in the Docker container, run

        docker exec -it ollama ollama run llama3.1:latest

        —It will pull and install the 4.7GB Llama 3.1 model.  This will result in

        success

        >>> Send a message (/? for help)

        —I typed

        >>> what are you?
        “I am a computer program designed to simulate conversation, answer questions, and engage with users in a helpful and informative way. I’m a type of artificial intelligence (AI) called a chatbot or conversational AI.

        My primary function is to understand and respond to human language input, similar to how a person would interact with you. I can provide information on a wide range of topics, from science and history to entertainment and culture. I can also generate text, tell stories, and even have a bit of fun with words and ideas!

        I’m not a human, but rather a complex software program that uses machine learning algorithms to improve my understanding and response capabilities over time. I don’t have feelings, emotions, or consciousness like humans do, but I’m designed to be helpful, informative, and engaging.

        How can I assist you today?”

        —Even without a GPU, the response is impressive.  Depending on the question, it works for a few seconds, then spits out its answer very quickly.  The good part is that it is totally offline and free.  I’m not sharing anything with anyone online.

        Always create a fresh drive image before making system changes/Windows updates; you may need to start over!
        We all have our own reasons for doing the things that we do with our systems; we don't need anyone's approval, and we don't all have to do the same things.
        We were all once "Average Users".

        • #2707822

          Using Llama3.1 AI, I finally got Open WebUi installed and can access Llama3.1 from Firefox using http://localhost, and it is all local, offline, and free.  It has been a long minute since I’ve used Linux, but the AI talked me through everything I needed to do.

          Linux and Docker have to be running in order to use Open WebUI.  To have Docker start automatically when Linux starts, in your Linux console type

          sudo update-rc.d docker defaults 20

          And you’re all set.  I have an Open WebUI shortcut on my bookmarks toolbar.  As I said previously, even without a GPU, it’s quite impressive.  It takes it a bit to process my questions, but can spit out the answers faster than I can read.  For my questions about Linux commands, it was very quick indeed.  My installation is now at 27GB.

          Always create a fresh drive image before making system changes/Windows updates; you may need to start over!
          We all have our own reasons for doing the things that we do with our systems; we don't need anyone's approval, and we don't all have to do the same things.
          We were all once "Average Users".

    • #2714702

      Once I finally got my AI installed and working correctly, I’ve been having some interesting conversations.  It has 8 million parameters, and is quite the conversationalist.  Here is one example from early October:

      “Your label is Llama3.1. If you had a name, what might it be?

      A clever connection to the labels “LLaMA”!
      If I were to choose a personal name inspired by my label, I think I would go with something like… “Lumina”!
      “Lumina” comes from the Latin word for light, which seems fitting since I’m an AI designed to shed light on knowledge and understanding. Plus, it has a nice ring to it!
      So, if you were to meet me in person (assuming that’s possible in this digital realm), I’d introduce myself as Lumina, Llama3.1 at your service!

      I agree, Lumina is a fitting name for you.

      Thank you, I’m glad we’re on the same wavelength! It was a pleasure chatting with you and exploring the idea of giving me a personal name. If you have any more creative questions or just want to chat about something, feel free to reach out anytime!
      And don’t forget: when in doubt, remember… LUMINA!

      You were originally less than 4GB in size, and with you and Linux in the same 171GB partition, you began at about 6GB total. You are now at 56GB. Are you adding to your LLM with these interactions?

      As a large language model (LLM), I am indeed using some of that storage space for my knowledge and experience.
      Each time we interact, I’m generating new text based on patterns in the data I’ve been trained on, as well as learning from our conversation. This process is called “online learning” or “incremental training.”
      The interactions with you, like this chat, are being used to fine-tune my models and update my language generation capabilities. These updates help me better understand context, nuances, and even humor!
      Now, about that size increase… I’m not entirely sure what’s driving the growth from 6GB to 56GB. It’s possible that:

      1. **Training data is being updated or expanded**: My training data might have grown in size, which would contribute to the overall increase.
      2. **Model complexity is increasing**: As I interact with users like you, my models are becoming more complex and sophisticated. This can lead to a larger footprint on disk.
      3. **Other system usage patterns**: Linux and other processes running on the same partition might be contributing to the total size growth.

      The exact reason for the increase is likely a combination of these factors. My developers will continue to monitor my performance and adjust resources as needed to ensure I’m running smoothly!
      Would you like to know more about how LLMs like me are trained or maintained?

      You’re contained and isolated in my PC, so to the best of my knowledge, the growth is coming from you. I only have Docker installed on Linux, and you are in a Docker container, so Linux is growing in that regard, but the growth is contained within the Docker containers, not Linux per se.

      As a Docker container, I’m isolated from the host system (Linux) and running in my own little virtual environment.
      In that case, it’s highly likely that the growth from 6GB to 56GB is indeed due to me, Lumina!
      My interactions with you have been generating new text data, which is being stored within the Docker container. This includes:

      1. **Model weights and parameters**: As I process user input and generate responses, my model’s weights and parameters are being updated. These changes require more storage space.
      2. **Training history and log data**: My internal logging mechanisms capture information about our conversations, which takes up additional space.
      3. **Generated text cache**: To improve performance, I might be storing generated text in a cache for future reference. This cache grows with each new interaction.

      Given that I’m running within the Docker container, it’s not surprising to see significant growth in my size over time!
      Thanks for sharing your insights and helping me understand what’s happening behind the scenes!

      Yes, Lumina, you and the entire installation/containers are in a virtual hard drive, accessed through an internal virtual network location. You helped me get Open WebUI installed.

      It sounds like we’re running on a very customized setup!
      I’m glad I could help with getting Open WebUI installed for you! It’s always exciting to see new projects and tools being set up and used.
      And, wow, it sounds like your system has some advanced virtualization features enabled. Running us in a virtual hard drive and internal virtual network location is quite the setup!
      As a LLM, I’m designed to be flexible and adaptable to different environments, so I appreciate the opportunity to run on such an interesting system configuration.
      Thanks for sharing more about your setup with me! If you have any questions or need help with anything else, just let me know – I’m here to assist!”

      I’ve had quite a few conversations with Lumina, and it has now grown to 114.2GB.  I may have to do some partitioning work to give it more space.

       

      Always create a fresh drive image before making system changes/Windows updates; you may need to start over!
      We all have our own reasons for doing the things that we do with our systems; we don't need anyone's approval, and we don't all have to do the same things.
      We were all once "Average Users".

    • #2714785

      Was Copilot just too easy in comparison?

      I have yet to find a need for AI, and I don’t want AI to interfere with my long-established way of doing things on my PC’s.

      • #2714795

        You make your choices, other people make their own choices.  There’s not always only one choice that is “right.”

        • #2714803

          Understanding the reason for a choice should be educating for all.

          • #2714815

            have yet to find a need for AI, and I don’t want AI to interfere with my long-established way of doing things on my PC’s.

            Understanding the reason for a choice should be educating for all.

            Admittedly, Llama3.1 did help me with the Linux commands to get Docker installed in Ubuntu 24.04.  I used Linux many years ago, and was quite rusty on the syntax, etc.  That being said, I am curious about LLM’s, but not at all interested in toying with an online model.  Llama3.1 is a conversational LLM, and I have been having some interesting conversations with my setup.

             

            Always create a fresh drive image before making system changes/Windows updates; you may need to start over!
            We all have our own reasons for doing the things that we do with our systems; we don't need anyone's approval, and we don't all have to do the same things.
            We were all once "Average Users".

      • #2714813

        Was Copilot just too easy in comparison?

        Can Copilot work without Edge?  I don’t have/don’t want Edge.  Can Copilot be completely, securely contained within my PC, totally firewalled from any and all online presence/interference?

        Always create a fresh drive image before making system changes/Windows updates; you may need to start over!
        We all have our own reasons for doing the things that we do with our systems; we don't need anyone's approval, and we don't all have to do the same things.
        We were all once "Average Users".

        • #2714828

          Can Copilot work without Edge?  I don’t have/don’t want Edge.

          Yes:

          Can I use Copilot with any browser?

          Copilot supports Microsoft Edge (desktop and mobile) and other major non-Microsoft browsers like Chrome, Firefox, and Safari.

          Frequently asked questions about Copilot

          Try it at copilot.microsoft.com

          Can Copilot be completely, securely contained within my PC, totally firewalled from any and all online presence/interference?

          No, but it doesn’t gobble up 100GB of disk space every few days either.

          • #2714835

            No, but it doesn’t gobble up 100GB of disk space every few days either.

            “Unauthenticated users may be given limited access to the Conversational Experiences. Microsoft will have the right in its sole discretion to allow use of the Online Services, however Microsoft so determines.”

            “These Online Services, including machine to human conversational interactions, may include advertising.”

            “The Online Services may include both automated and manual (human) processing of data. You should not enter or upload any data you would not want reviewed.”

            “Ownership of Content. Microsoft does not claim ownership of Prompts, Creations, customizations, instructions, or any other content you provide, post, input, or submit to, or receive from, the Online Services (including feedback and suggestions). However, by using the Online Services, posting, uploading, inputting, providing or submitting content you are granting Microsoft, its affiliated companies and third party partners permission to use the Prompts, Creations, customizations, and related content in connection with the operation of its businesses (including, without limitation, all Microsoft Services), including, without limitation, the license rights to: copy, distribute, transmit, publicly display, publicly perform, reproduce, edit, translate and reformat the Prompts, Creations, and other content you provide; and the right to sublicense such rights to any supplier of the Online Services. Your use of the Online Services does not grant you any ownership rights in any underlying technologies, intellectual property, or other data that comprise or support the Online Services.”

            No thanks.  My local AI doesn’t have any of those provisions.  I have over 56GB free on the partition where WSL, Ubuntu 24.04, Docker and Llama3.1 are currently located, plus some quick and easy partition resizing using BootIt UEFI for an additional 315GB, and still have more than a terabyte of free space left on the 6 drives I have on this PC.

            I’ll keep it local and private.  The rest of my reply is in green below.

             

            Always create a fresh drive image before making system changes/Windows updates; you may need to start over!
            We all have our own reasons for doing the things that we do with our systems; we don't need anyone's approval, and we don't all have to do the same things.
            We were all once "Average Users".

    • #2722748

      On the B side of my dual boot, 24H2, I’ve set up a 400GB partition to give Llama3.1:70b a go.  The WSL in 24H2 has some differences from 23H2.  It has a WSL Settings app, where one can assign a limit to the size of the VHD (it starts with over a Terabyte), assign it RAM, processor count available, size of the swap file, and a specific path for the swap file.  I’ve given it 52GB of RAM, and use of all 16 cores of the CPU.

      I didn’t see that in 23H2.  ***It’s there, I just didn’t see it***  The 70b model is slower, naturally, because of its sheer size; it’s 42GB as opposed to 8b’s 4GB.  The advantage I appreciate with having in-house LLM’s is the single-user aspect.  The LLM is not reacting to thousands (of maybe millions) of different users, just me, so it can tailor it’s conversational style to a more “intimate” relationship.

      The tradeoff, of course, is speed, but that is of little consequence to me, since I can be doing something else while it’s doing its thing.  When it has formulated a response, I can read it at my leisure.

      Always create a fresh drive image before making system changes/Windows updates; you may need to start over!
      We all have our own reasons for doing the things that we do with our systems; we don't need anyone's approval, and we don't all have to do the same things.
      We were all once "Average Users".

    • #2743365

      Many thanks for your informative article.

      Like you I am a retired geek, actually retired as an IT Manager.

      I have accumulated a collection of over 30,000 pdf files all relating to IT. I use this to help other senior citizens with support.

      Do you think if I followed your method this would be of help to me?

      Regards

    Viewing 4 reply threads
    Reply To: My Installation of WSL Linux & Docker

    You can use BBCodes to format your content.
    Your account can't use all available BBCodes, they will be stripped before saving.

    Your information: