[-] TheOtherJake@beehaw.org 16 points 1 year ago

I won't touch the proprietary junk. Big tech "free" usually means street corner data whore. I have a dozen FOSS models running offline on my computer though. I also have text to image, text to speech, am working on speech to text, and probably my ironman suit after that.

These things can't be trusted though. It is just a next word statistical prediction system combined with a categorization system. There are ways to make an LLM trustworthy, but it involves offline databases and prompting for direct citations, these are different from Chat prompt structures.

118
submitted 1 year ago* (last edited 1 year ago) by TheOtherJake@beehaw.org to c/lgbtq_plus@beehaw.org

Much love.

[-] TheOtherJake@beehaw.org 21 points 1 year ago

Oobabooga is the main GUI used to interact with models.

https://github.com/oobabooga/text-generation-webui

FYI, you need to find checkpoint models. In the available chat models space, naming can be ambiguous for a few reasons I'm not going to ramble about here. The main source of models is Hugging Face. Start with this model (or get the censored version):

https://huggingface.co/TheBloke/llama2_7b_chat_uncensored-GGML

First, let's break down the title.

  • This is a model based in Meta's Llama2.
  • This is not "FOSS" in the GPL/MIT type of context. This model has a license that is quite broad in scope with the key point stipulating it can not be used commercially for apps that have more than 700 million users.
  • Next, it was quantized by a popular user going by "The Bloke." I have no idea who this is IRL but I imagine this is a pseudonym or corporate alias given how much content is uploaded by this account on HF.
  • This model is based on a 7 Billion parameter dataset, and is fine tuned for chat applications.
  • This is uncensored meaning it will respond to most inputs as best it can. It can get NSFW, or talk about almost anything. In practice there are still some minor biases that are likely just over arching morality inherent to the datasets used, or it might be coded somewhere obscure.
  • Last part of the title is that this is a GGML model. This means it can run on CPU or GPU or a split between the two.

As for options on the landing page or "model card"

  • you need to get one of the older style models that have "q(numb)" as the quantization type. Do not get the ones that say "qK" as these won't work with the llama.cpp file you will get with Oobabooga.
  • look at the guide at the bottom of the model card where it tells you how much ram you need for each quantization type. If you have a Nvidia GPU with the CUDA API, enabling GPU layers makes the model run faster, and with quite a bit less system memory from what is stated on the model card.

The 7B models are about like having a conversation with your average teenager. Asking technical questions yielded around 50% accuracy in my experience. A 13B model got around 80% accuracy. The 30B WizardLM is around 90-95%. I'm still working on trying to get a 70B running on my computer. A lot of the larger models require compiling tools from source. They won't work directly with Oobabooga.

14

My main reason for playing with offline AI right now is to help me get further into the Computer Science curriculum on my own. (disabled/just curious)

I have seen a few AI chat characters with highly detailed prompts that attempt to keep the LLM boxed into a cosplay character. I would like to try to create fellow students in a learning curriculum. I haven't seen anything like this yet, but maybe someone else here has seen this or has some helpful tips. I would like to prompt a character to not directly use programming knowledge from its base tokens and only use what is available in a Lora, or a large context, or a langchain database. I would like to have the experience of learning along side someone to talk out ideas when they have the same amount of information as myself. Like, I could grab all the information for a university lecture posted online and feed it to the AI, watch and read the information myself, and work through the quizzes or question anything I do not understand with the answers restricted to my own internal context region.

[-] TheOtherJake@beehaw.org 18 points 1 year ago

Hey there Lionir. Thanks for the post. Can the Beehaw team please look into copying or getting the creator of this bot to work here? https://lemmy.world/u/PipedLinkBot@feddit.rocks

I think the person that created that bot is somehow connected to the piped.video project. I know the whole privacy consciousness thing isn't for everyone, but this bot's posts are quite popular elsewhere on Lemmy.

FYI, the main reason to use piped.video links is that it is setup as an alternative front end for YT that automatically routes all users through a bunch of VPNs to help mitigate Alphabet's privacy abuses and manipulation.

[-] TheOtherJake@beehaw.org 15 points 1 year ago

Google is broken because AI is making it obsolete. I bet in 10 years google will be a historical footnote.

103

I just got Oobabooga running for the first time with Llama-2, and have Automatic1111, and ComfyUI running for images. I am curious about ML too but I don't know where this start with that one yet.

For the uninitiated, all of these tools are running offline open source (or mostly) models.

2
submitted 1 year ago by TheOtherJake@beehaw.org to c/news@beehaw.org
6
submitted 1 year ago by TheOtherJake@beehaw.org to c/news@beehaw.org
5

I encountered someone saying, "I have no problems with a person's sexual orientation and choice, I have a problem with anyone being openly sexual or flaunting their sexuality in front of me regardless of their choice of orientation."

I am a card carrying atheist. I was raised in one of the worst fundamental christian extremist groups and now live in near isolation from abandoning it nearly 10 years ago. All sexuality was bottled in my life and surroundings. This is still my comfort zone. A part of me wants to hold on to a similar ethos as the person I mentioned above, but I feel like I'm not very confident it is the right inner philosophical balance either.

I'm partially disabled now, so this is almost completely hypothetical. I am honestly looking to grow in my understanding of personal space and inner morality as it relates to others. Someone enlighten me please. Where does this go, what does it mean to you?

6
submitted 1 year ago* (last edited 1 year ago) by TheOtherJake@beehaw.org to c/greenspace@beehaw.org

I'm looking for the farmer's almanac, anecdotal type of information more than anything scientific or generalized.

Specifically, I'm curious about typical behavior in Southern California coastal regions, and really, the micro climate within a few miles of the Pacific around the Los Angeles Basin, but I think that is a bit too specific of a question for a small community here.

I am developing an increased allergic reaction to stings. I have probably gotten around 2-3 dozen stings while riding a bike for the last 2 decades. I have mostly limited myself to riding in the last hour or two of daylight which seems to avoid bees. I'm curious if there are patterns of predictable/probabilistic inactivity other than the day/night cycle.

2

Like we're not triple-A machine possessors at this point. A friend and I played in the era of the original Age of Empires, and StarCraft; Worms, and Dune. We were core SNES-PS2 era. We were never the ultra competitive hotkey speed run strategy types, but just played for fun.

Anyone out there in your late 30s to early 40s that have managed to connect to old friends despite long distances, what are you playing now?

[-] TheOtherJake@beehaw.org 15 points 1 year ago

It could also be a way to shake out any elements of the Russian military or other paramilitary groups that are looking for ways to get out of Ukraine. Like, maybe the attacks on Wagner coming from the Russian army are rogue elements where Wagner is obviously the most effective target for friendly fire.

126
submitted 1 year ago* (last edited 1 year ago) by TheOtherJake@beehaw.org to c/news@beehaw.org

No paywall or permissions nonsense

[-] TheOtherJake@beehaw.org 24 points 1 year ago

Rule number one of buying a new car: get the dealer to disconnect the modem.

Cars should be entirely open source by government regulation. All software should be public and the manufacturer should be required to host and maintain a public toolchain that can reproduce the software and any revisions made. All of this should also get mirrored by the library of Congress and made publicly available as a second source indefinitely. This is about ownership. Digital rights are never okay to reserve. If I do not own everything I am only renting from the real owner. Proprietary goods are theft of ownership. It really is that simple.

[-] TheOtherJake@beehaw.org 20 points 1 year ago

"Hi Karen , this is HR. You can now log anonymous complaints about IT, by logging into this external website with your company credentials. We provide this for your security because IT is able to monitor in network communication."

[-] TheOtherJake@beehaw.org 14 points 1 year ago

OP is aware that the term hackers had, and still has, a completely different meaning in the community it originated from. A real hacker is someone with the skill to modify a codebase to suit their needs. The corruption of the term by corporate media is an attempt to steal the fundamental human right to share one's digital work freely by labeling these people and their work as criminal. Hacking means modifying source code to suit your specific needs. It could be used to refer to the efforts of someone learning to become a kernel developer or it could be someone modifying the software source code in ways that are not suitable to share with others, such as features that may be incompatible with some systems or features.

Cracking has long been proposed by hackers as a more appropriate term describing a bad actor attempting to gain unauthorized access to a system.

It seems pedantic at first and in my simple explanation here too, but if you look into the details, philosophy, politics, and people where this term originated from, then look at the opposition and how they profited, and finally look at the state of current society and the internet, you will likely see this as, at least mildly, offensive. Like, I still misuse the term as if it has dual meanings, but anyone using it correctly gets my attention right away.

[-] TheOtherJake@beehaw.org 20 points 1 year ago* (last edited 1 year ago)

They need to hit the final nail on the head. All smart phones sold in Europe must have fully documented and open source hardware including the entire chipset, all peripherals, and the modem, with all registers and interfaces documented, the full API, and all programing documentation along with a public toolchain that can reproduce the software as shipped with the device and updated with any changes made to future iterations as soon as the updated software is made available.

This law would make these devices lifetime devices, if you choose; as in your lifetime. It would murder the disposable hardware culture, and it should happen now. Moore's law is dead. The race is over.

[-] TheOtherJake@beehaw.org 17 points 1 year ago

...officials are working to get a remotely operated vehicle that can reach a depth of 6,000 meters (about 20,000 feet) to the site as soon as possible.

The 5-person submersible, named Titan, is capable of diving 4,000 meters or 13,120 ft. “with a comfortable safety margin,” OceanGate said in its filing with the court.

but...after looking up on Wikipedia

...a wreck that lies over 12,000 feet (3,700 m) below the surface...

[-] TheOtherJake@beehaw.org 32 points 1 year ago

There are several vulnerabilities in bootloaders that have not been fixed. Namely, there is an entire tiny operating system that is used to initialize the processor before the main bootloader begins. Then the bootloader creates a bunch of handles to control the hardware, and hands them over to the operating system kernel. The i-core/ryzen processors include the second generation of this tiny underlying operating system that runs before everything else. This tiny operating system was originally marketed as a way to remotely monitor and troubleshoot data center servers, but this is a very weak and flawed marketing strategy. The way this system runs before everything else, it has root/admin privileges and access that supersede everything that comes after it. A bad actor accessing this system is absolutely game over for all hardware including the bootloader itself. Well this tiny operating system is tied to the microcode for the processor generation.

If you know anything about old computers that had a bunch of boards and chips inside the case, modern computers still have all of these chips and systems, but they are all integrated into just a few chips. These systems are still complicated and have a certain way they must be powered up and initialized so that each system begins in a specific state along a long chain. The "microcode" in a modern computer is really just a bunch of "software" that controls the order that the hardware is brought online. In the i-core/ryzen generation of hardware the microcode is proprietary and copyright protected. This is a way to get around many x86 patents expiring. It has long been speculated that the tiny operating system is also a back door for governments as it can completely own any system regardless of encryption or any other security measures.

There is a way to mostly disable this tiny operating system but there is no way to monitor or confirm its activity at run time. OpenSIL is like having access to the control room of this tiny operating system for the first time. It means it is now possible to completely secure and verify the state of a system. There is no security in obscurity. OpenSIL is the removal of a major failed attempt at security through obscurity.

Ultimately, at the most fundamental level, openSIL means full ownership over your hardware. I can buy AMD, but can only rent Intel. Intel keeps ownership of this tiny little corner of the hardware and they have done a terrible job of managing what they own. When faced with that buying choice the outcome should be obvious, assuming you are able to run the software that can take advantage of this. It will take at least a few months, but there should soon be a completely open source version of Coreboot that will use openSIL. Hope this helps.

view more: next ›

TheOtherJake

joined 1 year ago