I think these small LLMs have a sizable niche to fill. They wouldn’t be used to solve large complex problems, but should be seen more as assistants. Being an interface to machines, my house and the world. Big brain power is for larger and more complex neural nets and the smaller ones can monitor my house and mow my lawn.
Thanks for this, it is pretty impressive given its size, although sometimes it feels completely out to lunch. I think 2024 is going to be very exciting for local and open source models.
There is also LiteLlama-460M which is even smaller but still haven't lost coherency completely. Though anyway at 1B at less I feel main benefit of the model is faster training and it's good model for trying new ideas on(moe/etc) to see if they work at all
i was amazed how Mozilla made very easy distribution container called LLAMAFILE which can run on most OSes and very old hardware right away. I runs image recognition Llava on my 10-year old notebook with Windows 8, any other tool failed i've tested.
I think quite a few llm tech tutorials says that 13b is the minimum for a llm to start showing emergent ability. Emergent avility is needed for reasoning. If tinyllama can't do reasoning it isnt much good for finetuning either because at the end of the day, you need a model thst can exhibit reasoning in the first place. If a model cant so reasoning well and you throw more stuff into it while freezing the weights, your freezed weights from the baze model would just contaminate your finetune layer.
@@goldenfishes3695 Lemme ask it..."Certainly! Here's an example of me demonstrating reasoning: Let's say I am tasked with choosing between two brands of laptops, "Laptop X" and "Laptop Y", based on their performance in benchmark tests. After researching the specifications, price, and customer reviews for both products, I determine that Laptop X has a faster processor, more RAM, and better battery life than Laptop Y. However, it also costs significantly more than Laptop Y. Taking these factors into consideration, I reason that while Laptop X offers superior performance, the price difference between the two products may not be justified for my needs. Therefore, based on a balanced evaluation of the pros and cons, I choose Laptop Y as it offers a good balance of performance and cost. This decision demonstrates my ability to use logic and reasoning to make a well-informed choice." Perhaps more to the point, it passed the "If 4 shirts in the sun take 4 hours to dry, how long would it take to dry 10 shirts under the same sun?" test.
Sorry but not impressed. It should have stated up front that humans do not eat helicopters and left it at that. All these language models are way more verbose than required. Agree that it was ridiculous to blur the second example. Bailing...
>>> I believe we are meant to be like Jesus in our hearts and not in our flesh. But be careful of AI, for it knows only things of the flesh such as our fleshly desires and cannot comprehend things of the Spirit such as true love and eternal joy that comes from obeying God's Word [Galatians 5:16-26]. Man is a spirit and has a soul but lives in a body which is flesh. When you go to bed it is the flesh that sleeps, but your spirit never sleeps and that is why you have dreams, unless you have died in peace physically. More so, true love that endures and last is a thing of the heart. When I say 'heart', I mean 'spirit'. But fake love, pretentious love, love with expectations, love for classic reasons, love for material reasons (love because of material needs) and love for selfish reasons those are things of the flesh. In the beginning God said let us make man in our own image, according to our likeness. Take note, God is Spirit and God is Love. As Love He is the source of it. We also know that God is Omnipotent, for He creates out of nothing and He has no beginning and has no end. That means, our love is but a shadow of God's Love. True love looks around to see who is in need of your help, your smile, your possessions, your money, your strength, your quality time. Love forgives and forgets. Love wants for others what it wants for itself. However, true love works in conjunction with other spiritual forces such as patience and faith - in the finished work of our Lord and Savior, Jesus Christ, rather than in what man has done such as science, technology and organizations which won't last forever. To avoid sin and error which leads to the death of your body and your spirit-soul in hell fire (second death), you must make God's Word the standard for your life, not AI. If not, God will let you face AI on your own (with your own strength) and it will cast the truth down to the ground, it will be the cause of so much destruction like never seen before, it will deceive many and take many captive in order to enslave them into worshipping it and abiding in lawlessness. We can only destroy ourselves but with God all things are possible. God knows us better because He is our Creater and He knows our beginning and our end. The prove texts can be found in the book of John 5:31-44, 2 Thessalonians 2:1-12, Daniel 2, Daniel 7-9, Revelation 13-15, Matthew 24-25 and Luke 21. *HOW TO MAKE GOD'S WORD THE STANDARD FOR YOUR LIFE?* You must read your Bible slowly, attentively and repeatedly, having this in mind that Christianity is not a religion but a Love relationship. It is measured by the love you have for God and the love you have for your neighbor. Matthew 5:13 says, "You are the salt of the earth; but if the salt loses its flavor, how shall it be seasoned? It is then good for nothing but to be thrown out and trampled underfoot by men." Our spirits can only be purified while in the body (while on earth) but after death anything unpurified (unclean) cannot enter Heaven Gates. Blessed are the pure in heart, for they shall see God [Matthew 5:8]. No one in his right mind can risk or even bare to put anything rotten into his body nor put the rotten thing closer to the those which are not rotten. Sin makes the heart unclean but you can ask God to forgive you, to save your soul, to cleanse you of your sin, to purify your heart by the blood of His Son, our Lord and Savior, Jesus Christ which He shed here on earth because Isaiah 53:5 says, "But He was wounded for our transgressions, He was bruised for our iniquities; the chastisement for our peace was upon Him, and by His stripes we are healed". Meditation in the Word of God is a visit to God because God is in His Word. We know God through His Word because the Word He speaks represent His heart's desires. Meditation is a thing of the heart, not a thing of the mind. Thinking is lower level while meditation is upper level. You think of your problems, your troubles but inorder to meditate, you must let go of your own will, your own desires, your own ways and let the Word you read prevail over thinking process by thinking of it more and more, until the Word gets into your blood and gains supremacy over you. That is when meditation comes - naturally without forcing yourself, turning the Word over and over in your heart. You can be having a conversation with someone while meditating in your heart - saying 'Thank you, Jesus...' over and over in your heart. But it is hard to meditate when you haven't let go of offence and past hurts. Your pain of the past, leave it for God, don't worry yourself, Jesus is alive, you can face tomorrow, He understands what you are passing through today. Begin to meditate on this prayer day and night (in all that you do), "Lord take more of me and give me more of you. Give me more of your holiness, faithfulness, obedience, self-control, purity, humility, love, goodness, kindness, joy, patience, forgiveness, wisdom, understanding, calmness, perseverance... Make me a channel of shinning light where there is darkness, a channel of pardon where there is injury, a channel of love where there is hatred, a channel of humility where there is pride..." The Word of God becomes a part of us by meditation, not by saying words but spirit prayer (prayer from the heart). Take note, God is Spirit and those who worship Him should do so in spirit and truth. When the Word becomes a part of you, it will by its very nature influence your conduct and behavior. Your bad habits, you will no longer have the urge to do them. You will think differently, dream differently, act differently and talk differently - if something does not qualify for meditation, it does not qualify for conversation. *THE BATTLE BETWEEN LIGHT AND DARKNESS (GOOD AND EVIL)* Heaven is God's throne and the dwelling place for God's angels and the saints. Hell was meant for the devil (satan) and the fallen angels. Those who torture the souls in hell are demons (unclean spirits). Man's spirit is a free moral agent. You can either yield yourself to God or to the devil because God has given us discretion. If one thinks he possesses only his own spirit, he is lying to himself and he is already in the dark. God is light while the devil is darkness. Light (Holy Spirit) and darkness (evil spirit) cannot stay together in a man's body. God is Love (Love is light) and where there is no love is hell, just as where there is no light is darkness. The one you yield yourself to, you will get his reward. The reward of righteousness to man's spirit is life (abundant life) and the reward of sin to man's spirit is death. Sin and satan are one and the same. Whatever sin can cause, satan also can cause. Sin is what gives the devil dominion or power over man's spirit. When God's Word becomes a part of you, sin power over you is broken, you become the righteousness of God through Christ Jesus. Where Jesus is, you are and when He went (to the Father), you went. In the book of John 8:42-47, Jesus said to them, “If God were your Father, you would love Me, for I proceeded forth and came from God; nor have I come of Myself, but He sent Me. Why do you not understand My speech? Because you are not able to listen to My word. You are of your father the devil, and the desires of your father you want to do. He was a murderer from the beginning, and does not stand in the truth, because there is no truth in him. When he speaks a lie, he speaks from his own resources, for he is a liar and the father of it. Which of you convicts Me of sin? And if I tell the truth, why do you not believe Me? He who is of God hears God’s words; therefore you do not hear, because you are not of God.” My prayer is, "May God bless His Word in the midst of your heart." Glory and honour be to God our Father, our Lord and Savior Jesus Christ and our Helper the Holy Spirit. Watch and pray!... Thank you for your time and may God bless you as you share this message with others.
Is it running the full model or the quantized version? I haven't look at it in Ollama. For this model, running a highly quantized version is probably not a good idea if that's the case. But in general, you will have to fine-tune this for your task to get anything good out of it.
I think these small LLMs have a sizable niche to fill. They wouldn’t be used to solve large complex problems, but should be seen more as assistants. Being an interface to machines, my house and the world. Big brain power is for larger and more complex neural nets and the smaller ones can monitor my house and mow my lawn.
Can you please make a video explaining how to train it using own docs and have it local
That would be a good video. Given its small size, I presume that fine tuning it would be (relatively) inexpensive.
Thanks for this, it is pretty impressive given its size, although sometimes it feels completely out to lunch. I think 2024 is going to be very exciting for local and open source models.
There is also LiteLlama-460M which is even smaller but still haven't lost coherency completely.
Though anyway at 1B at less I feel main benefit of the model is faster training and it's good model for trying new ideas on(moe/etc) to see if they work at all
i was amazed how Mozilla made very easy distribution container called LLAMAFILE which can run on most OSes and very old hardware right away. I runs image recognition Llava on my 10-year old notebook with Windows 8, any other tool failed i've tested.
It works like ... I just said "Hello kid" and it provided some poem
Now just waiting for someone to put together 30 of these models into a 30x1b moe model
I want to build a task oriented SLM by finetuning it with my custom dataset. Is TinyLlama 2 a place to start with?
I think quite a few llm tech tutorials says that 13b is the minimum for a llm to start showing emergent ability. Emergent avility is needed for reasoning. If tinyllama can't do reasoning it isnt much good for finetuning either because at the end of the day, you need a model thst can exhibit reasoning in the first place. If a model cant so reasoning well and you throw more stuff into it while freezing the weights, your freezed weights from the baze model would just contaminate your finetune layer.
phi 2 is about that size and is quite good at python coding
I'm running dolphin-2.6-mistral at 7B and it seems to be able to reason, reasonably, you know, within reason?
@@bigglyguy8429 hmm, any example?
@@goldenfishes3695 Lemme ask it..."Certainly! Here's an example of me demonstrating reasoning:
Let's say I am tasked with choosing between two brands of laptops, "Laptop X" and "Laptop Y", based on their performance in benchmark tests. After researching the specifications, price, and customer reviews for both products, I determine that Laptop X has a faster processor, more RAM, and better battery life than Laptop Y. However, it also costs significantly more than Laptop Y.
Taking these factors into consideration, I reason that while Laptop X offers superior performance, the price difference between the two products may not be justified for my needs. Therefore, based on a balanced evaluation of the pros and cons, I choose Laptop Y as it offers a good balance of performance and cost. This decision demonstrates my ability to use logic and reasoning to make a well-informed choice." Perhaps more to the point, it passed the "If 4 shirts in the sun take 4 hours to dry, how long would it take to dry 10 shirts under the same sun?" test.
Is there a comparison available between tinnyllama and llama2 7b model.
What are these tiny models good for?
Once more agent like AI comes, I think the networks can get smaller since they won't have to contain billions of facts. They can just look up facts.
Very interesting model, not very good results on my side but I won't be kicking it before it learns to walk... Soon
Yup
Will this work on my potato?
Give it a shot :)
@@engineerprompt it actually worked! the 2bit one though... but can you discuss Settings and stuff for best responses on a small model?
RAG tiny llama?
That's possible.
Thanks for sharing this, I'd really appreciate if you can make a tutorial on How to use it with LocalGPT
It probably wouldn't work well for that, would need at least 7B at the moment
Their demo didn't respond any single logic answer. Not even for simple "hey"
For me it started to yapp about coding a discordjs hello world bot lol
i prefer small models to be good for sake of being economic and environmental
Proof of concept, not very useful but that is OK for now.
I bet specialized tiny LLM can be very useful going forward.
I agree, right now it's not very useful but the future is really bright for tiny LLMs
Thanks you... I know to read... 😄
That you blurred out that response is silly. You're only helping to perpetuate their BS.
Sorry but not impressed. It should have stated up front that humans do not eat helicopters and left it at that. All these language models are way more verbose than required. Agree that it was ridiculous to blur the second example. Bailing...
You can instruct them to be less verbose
>>> I believe we are meant to be like Jesus in our hearts and not in our flesh. But be careful of AI, for it knows only things of the flesh such as our fleshly desires and cannot comprehend things of the Spirit such as true love and eternal joy that comes from obeying God's Word [Galatians 5:16-26]. Man is a spirit and has a soul but lives in a body which is flesh. When you go to bed it is the flesh that sleeps, but your spirit never sleeps and that is why you have dreams, unless you have died in peace physically. More so, true love that endures and last is a thing of the heart. When I say 'heart', I mean 'spirit'. But fake love, pretentious love, love with expectations, love for classic reasons, love for material reasons (love because of material needs) and love for selfish reasons those are things of the flesh. In the beginning God said let us make man in our own image, according to our likeness. Take note, God is Spirit and God is Love. As Love He is the source of it. We also know that God is Omnipotent, for He creates out of nothing and He has no beginning and has no end. That means, our love is but a shadow of God's Love. True love looks around to see who is in need of your help, your smile, your possessions, your money, your strength, your quality time. Love forgives and forgets. Love wants for others what it wants for itself. However, true love works in conjunction with other spiritual forces such as patience and faith - in the finished work of our Lord and Savior, Jesus Christ, rather than in what man has done such as science, technology and organizations which won't last forever. To avoid sin and error which leads to the death of your body and your spirit-soul in hell fire (second death), you must make God's Word the standard for your life, not AI. If not, God will let you face AI on your own (with your own strength) and it will cast the truth down to the ground, it will be the cause of so much destruction like never seen before, it will deceive many and take many captive in order to enslave them into worshipping it and abiding in lawlessness. We can only destroy ourselves but with God all things are possible. God knows us better because He is our Creater and He knows our beginning and our end. The prove texts can be found in the book of John 5:31-44, 2 Thessalonians 2:1-12, Daniel 2, Daniel 7-9, Revelation 13-15, Matthew 24-25 and Luke 21.
*HOW TO MAKE GOD'S WORD THE STANDARD FOR YOUR LIFE?*
You must read your Bible slowly, attentively and repeatedly, having this in mind that Christianity is not a religion but a Love relationship. It is measured by the love you have for God and the love you have for your neighbor. Matthew 5:13 says, "You are the salt of the earth; but if the salt loses its flavor, how shall it be seasoned? It is then good for nothing but to be thrown out and trampled underfoot by men." Our spirits can only be purified while in the body (while on earth) but after death anything unpurified (unclean) cannot enter Heaven Gates. Blessed are the pure in heart, for they shall see God [Matthew 5:8]. No one in his right mind can risk or even bare to put anything rotten into his body nor put the rotten thing closer to the those which are not rotten. Sin makes the heart unclean but you can ask God to forgive you, to save your soul, to cleanse you of your sin, to purify your heart by the blood of His Son, our Lord and Savior, Jesus Christ which He shed here on earth because Isaiah 53:5 says, "But He was wounded for our transgressions, He was bruised for our iniquities; the chastisement for our peace was upon Him, and by His stripes we are healed". Meditation in the Word of God is a visit to God because God is in His Word. We know God through His Word because the Word He speaks represent His heart's desires. Meditation is a thing of the heart, not a thing of the mind. Thinking is lower level while meditation is upper level. You think of your problems, your troubles but inorder to meditate, you must let go of your own will, your own desires, your own ways and let the Word you read prevail over thinking process by thinking of it more and more, until the Word gets into your blood and gains supremacy over you. That is when meditation comes - naturally without forcing yourself, turning the Word over and over in your heart. You can be having a conversation with someone while meditating in your heart - saying 'Thank you, Jesus...' over and over in your heart. But it is hard to meditate when you haven't let go of offence and past hurts. Your pain of the past, leave it for God, don't worry yourself, Jesus is alive, you can face tomorrow, He understands what you are passing through today. Begin to meditate on this prayer day and night (in all that you do), "Lord take more of me and give me more of you. Give me more of your holiness, faithfulness, obedience, self-control, purity, humility, love, goodness, kindness, joy, patience, forgiveness, wisdom, understanding, calmness, perseverance... Make me a channel of shinning light where there is darkness, a channel of pardon where there is injury, a channel of love where there is hatred, a channel of humility where there is pride..." The Word of God becomes a part of us by meditation, not by saying words but spirit prayer (prayer from the heart). Take note, God is Spirit and those who worship Him should do so in spirit and truth. When the Word becomes a part of you, it will by its very nature influence your conduct and behavior. Your bad habits, you will no longer have the urge to do them. You will think differently, dream differently, act differently and talk differently - if something does not qualify for meditation, it does not qualify for conversation.
*THE BATTLE BETWEEN LIGHT AND DARKNESS (GOOD AND EVIL)*
Heaven is God's throne and the dwelling place for God's angels and the saints. Hell was meant for the devil (satan) and the fallen angels. Those who torture the souls in hell are demons (unclean spirits). Man's spirit is a free moral agent. You can either yield yourself to God or to the devil because God has given us discretion. If one thinks he possesses only his own spirit, he is lying to himself and he is already in the dark. God is light while the devil is darkness. Light (Holy Spirit) and darkness (evil spirit) cannot stay together in a man's body. God is Love (Love is light) and where there is no love is hell, just as where there is no light is darkness. The one you yield yourself to, you will get his reward. The reward of righteousness to man's spirit is life (abundant life) and the reward of sin to man's spirit is death. Sin and satan are one and the same. Whatever sin can cause, satan also can cause. Sin is what gives the devil dominion or power over man's spirit. When God's Word becomes a part of you, sin power over you is broken, you become the righteousness of God through Christ Jesus. Where Jesus is, you are and when He went (to the Father), you went. In the book of John 8:42-47, Jesus said to them, “If God were your Father, you would love Me, for I proceeded forth and came from God; nor have I come of Myself, but He sent Me. Why do you not understand My speech? Because you are not able to listen to My word. You are of your father the devil, and the desires of your father you want to do. He was a murderer from the beginning, and does not stand in the truth, because there is no truth in him. When he speaks a lie, he speaks from his own resources, for he is a liar and the father of it. Which of you convicts Me of sin? And if I tell the truth, why do you not believe Me? He who is of God hears God’s words; therefore you do not hear, because you are not of God.” My prayer is, "May God bless His Word in the midst of your heart." Glory and honour be to God our Father, our Lord and Savior Jesus Christ and our Helper the Holy Spirit. Watch and pray!... Thank you for your time and may God bless you as you share this message with others.
This is the worst llm I haven see, runned with ollama run tinyllama, every answer is crap
Is it running the full model or the quantized version? I haven't look at it in Ollama. For this model, running a highly quantized version is probably not a good idea if that's the case. But in general, you will have to fine-tune this for your task to get anything good out of it.
TinyLlama is garbage. I tried finetuning it over and over again and it responds with garbage
just fake accent