I think doing the calculations at full precision (FP16) is a waste. You should try somewhere between the Q4_K_M version to Q6_K (or at least Q8_0, that's supposed to the same quality as FP16). That way it should be considerably faster... At least twice as fast.

(The GGUF page of that model has a list of recommended quantization levels.)

Nice! KoboldCpp is also my software of choice. It's easy to install, all-in-one and has a good amount of features.

What kind of model size do you use to arrive at 1token/s? I'm in the same ballpark. Though my old desktop PC is a bit faster than my laptop. Probably because it has dual-channel memory and doesn't throttle.

I think that's the point where it gets usable. At least for consecutive chat. If I feed in longer text, or KoboldCpp decides to recalculate large portions of the context, it'll be several minutes for me until I get a reply. And that's less fun for use-cases like dialougue.

!chatbotsnsfw@lemmynsfw.com is related if you're into ERP with artificial intelligence. It's low activity there but a few people read the posts. I don't know any other places on the Fediverse. I read people meet on Reddit and then go to some Discord servers or something like that.

Nice, thanks for sharing and posting something in this low-frequency community 😅

Is there a way to pull out the character cards / prompts from poe? Bc I use a different service and I'm more used to downloading characters from characterhub.org which lets you see the full text.

• L3-8B-Stheno ( GGUF ) seems quite nice, worthy successor to Fimbulvetr
• also L3-8B-Lunaris seems really good

Some random news articles, blog posts I've found with other people's opinions and experiences:

• The rise of AI girlfriends — why your ex’s next girlfriend might be a chatbot | Same article on MSN
• A Psychologist Reveals 2 Dangers Of Falling For An AI Girlfriend
• AI girlfriends are ruining an entire generation of men
• Why people are falling in love with AI chatbots (Podcast)
• The Impact of Artificial Intelligence on Our Encounters
• AI ‘companions’ promise to combat loneliness, but history shows the dangers of one-way relationships
• The oddity — and sadness? — of AI companions
• Character.ai: Young people turning to AI therapist bots
• KI-Freundinnen: Sie tun alles, was er sagt (German, Lemmy discussion)

Papers:

• Ideal technologies, ideal women: AI and gender imaginaries in Redditors’ discussions on the Replika bot girlfriend (August 19, 2022 | still missing on sci-hub as of now)

So... I've tested both models and I think I can see what you like about them. And -wow- I didn't get to try 70B models before and it's really a step up. With smaller models it's more mixed, sometimes they get a complex concept, sometimes they don't. And seems the 70B model is able to pick on a good amount of more complexity and it has the intelligence to understand more things and is then able to go in some proper direction. At least more often.

I'm not entirely sure if I can make good use of my new information... Writing erotic literature really isn't that easy. I've been tinkering around with AI assisted storywriting for some time now, and I never got good results that I'd like to share. I mean it can write simple smut... And regarding that: A quick thanks to you. I read your other comment over at !asklemmynsfw and I think I agree with your opinion on erotic stories. I've now included a specific instruction to my prompt to balance the story more, alike you said there. Focus on a good story, make it tingle but the porn has to be the icing on the cake. For now I've also instructed it to contrast both things, have a story that raises questions and is intellectual and provide a stark contrast with immersive acts and graphic description... "The skillful combination of both aspects is what makes this story excel." Let's see what the LLM can do with that instruction...

But storywriting really isn't easy. Even the 70B model is far from perfect. And to this point I didn't find a single model that can do everything. Some of them are intelligent but not necessarily good for stories. Some of them seem to have been trained on stories, they get the language right for such a thing, some overdo it. And not every model can write lewd stories, it's really obvious if a model has seen some erotic literature or simple smut or no such stories and just writes one or two abstract sentences, summarizing it, because it's never seen more detailed descriptions. And there is the pacing... I think local LLMs are still far away from being able to write stories on their own. Some consistently write like 10 paragraps and call it a novel. Almost all of them brush over things that would be interesting to explore, instead they focus on some other scene that's kind of boring. They write meaningless dialogue that would be alright if I was casually talking to a chatbot and role-playing my every-day life, but not very interesting here. They miss important stuff and make up random details later on. I mean half the models don't have a clue what is interesting to write and what can be skipped or summarized.

Another issue is trying to wrap up things (early) or pushing towards the end. Or doing super obvious plot twists. Sometimes this makes me laugh. But they're also very creative. I like that. Inbetween the (sometimes) bad writing there's often some interesting ideas or crazy creativity, things I wouldn't have thought of. Or other gems, single sentences that really get something on point.

I'm still exploring. I've tried different approaches, laying down a rough concept of a setting and then letting it do it. I've also tried being more methodical and giving it to them more like a homework assignment. Come up with ideas to explore... then with several plot ideas... then give critique to themselves, pick one and revise it... Come up with the characters... Then the main story arc, subplots, twists and important scenes, write down the table of contents and chapter names to get a structure for the novel... And then start the actual writing with all of that information laid out.

I think that's yielded the best results so far. I'm positive I'll get to at a point where I like the results enough to upload them. And write a guide how exactly I did that. Currently it's more or less me writing 80%, pausing the LLM after every second sentence, revising that and constantly pushing the story towards a better direction and fighting the level of detail the LLM deemed appropriate. I think I will get better. Turns out I've been using the wrong models anyways and relied too much on Psyfighter and such, which might be great for role-play dialogue. But with my recent test it turns out I don't really like their output when it gets to storywriting.

Edit: Yeah, and one thing more: It came up with a nice plot which I liked and explored further. And at some point the AI cited the 2018 science fiction movie it got all the ideas from 😆 That really made me laugh. Seems some of my ideas aren't that original. But getting some recommendations is nice, I'll just skip writing the story myself and watch the movie then.

Thanks! Yeah, you were kind enough to include a bit of extra info in your previous posts. Your stories are somewhat specific and complex. I figured if you like a model... it has to be 'intelligent' enough to keep track...

I wonder if I also like that model for my purposes. I'm not sure if I can run the 70B model, I'd have to spin up a runpod cloud instance for that. But I'll try the FlatDolphinMaid 8x7B tomorrow.

You're right. (Good) AI storywriting and finding good models and settings isn't easy. I also discarded models and approaches because the prompt (or settings) I used didn't work that well and it later turned out I should have done more testing and got to like that model, all it needed was a different wording or better settings.

And some models have unique quirks or style or things they excel at... Which might skew expectations when switching to a different model.

Hehe, glad it turned out quite alright for you. I guess there might be a happy ending to that story/letter. And now I get the context of your writing.

I also really like people who are direct an honest. If it were up to me, we could skip most of the clues and hinting at things. I mean if we're trying to convey something to another human being we can make a deliberate choice to either make it simple and just say what's on our mind, or make it a riddle for them to solve. I like the first approach and I think it would be to the benefit of everyone if people chose to do it that way.

Ultimately your situation is a bit different anyways... I wouldn't dare to ask a random person for sex, it would be wrong. But if you're already sending dick pics and talking about your cock... ¯_(ツ)_/¯ You can probably skip the social etiquette and speak your mind. They're the right person to be honest with, or they would have ran away way earlier. (My opinion)